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1.  INTRODUCTION: 


Veterans  present  to  primary  care  providers  (PCPs)  with  posttraumatic  stress  (PTS) 
symptoms  because  many  are  resistant  to  specialty  mental  health  care.  Most  PCPs 
have  not  been  trained  to  assess  for  and  manage  symptoms  of  PTS  or  motivate 
Veterans  to  engage  in  treatment.  This  can  result  in  missed  opportunities  to  intervene  to 
prevent  chronic  mental  and  physical  health  problems.  The  project  aims  are  to:  (1) 
iteratively  design  a  new  web-based  PTS  and  Motivational  Interviewing  training  for 
PCPs  using  Virtual  World  technology  to  enhance  interactivity;  (2)  implement  a  robust 
evaluation  including  a  randomized  control  trial  for  clinically  valid  outcome 
measurement;  (3)  conduct  a  summative  evaluation  to  inform  national  “scale-up” 
dissemination  and  implementation.  The  project  will  produce  a  deliverable  that  will 
improve  access  to  quality  clinical  care  for  our  warfighters  suffering  with  PTS.  This 
report  shares  progress  made  during  Year  1  of  the  project. 


2.  KEYWORDS 


Virtual  reality;  PTSD;  medical  education;  virtual  training;  curriculum  development; 
motivational  interviewing 


3.  ACCOMPLISHMENTS: 

Year  1  of  the  project  was  dedicated  to  the  developmental  formative  evaluation 
and  curriculum  development  for  the  Virtual  World  PTSD  and  Motivational  Interviewing 
training.  The  Principal  Investigator,  Project  Coordinator,  and  other  Co-Investigators 
advanced  the  development  of  the  training  curriculum,  focusing  on  building  content  that 
is  consistent  with  the  overall  learning  objectives.  Concurrently,  the  vendors  at  Heyden 
Ty  provided  guidance  for  virtualizing  elements  of  the  curriculum  to  build  an  engaging 
and  impactful  training.  Additionally,  CNDG  staff  hosted  Second  Life  orientation  sessions 
to  acquaint  project  team  members  with  the  technical  functionalities  of  a  virtual  world  and 
to  ground  the  team’s  curriculum  work  in  the  potential  capabilities  of  a  virtual  learning 
environment. 

In  terms  of  human  subjects,  continuing  review  of  the  research  protocol  was 
submitted  and  approved.  The  Project  Coordinator  developed  procedures  for  conducting 
and  tracking  recruitment,  interviews,  and  analysis.  The  qualitative  researchers  on  the 
project  team  authored  a  semi-structured  interview  guide  and  met  weekly  to  plan  for 
stakeholder  recruitment,  as  well  as  how  the  data  collected  can  inform  the  training 
curriculum.  In  total,  11  interviews  were  conducted  with  primary  care  providers  and 
healthcare  leadership.  The  data  was  analyzed  and  summarized  into  a  technical  report, 
which  the  team  hopes  to  publish  as  a  manuscript  in  the  next  project  year. 

Much  progress  was  made  surrounding  the  development  of  the  training  and 
curriculum.  Heyden  Ty,  the  project’s  virtual  world  curriculum  experts,  developed 
storyboards  documenting  the  different  segments  of  the  virtual  world  training.  Then, 
findings  from  the  interviews  with  PCPs  and  leadership  were  incorporated  into  the 
storyboards  to  produce  a  more  refined  outline  for  the  virtual  world  developers  at  Chant 
Newall  Development  Group  (CNDG).  CNDG  produced  a  video  overview  of  the  virtual 
environment,  including  beta  versions  of  avatars,  objects,  and  functionality.  They 
reviewed  the  storyboards  for  feasibility  and  content,  and  will  dedicate  Quarter  1  of  Year 
2  of  the  project  to  carrying  out  the  design  of  the  training. 

What  were  the  major  goals  of  the  project? 

The  table  below  reflects  Month  1,  October  201 5,  through  Month  12,  September  2016. 


Task 

Projected 

Month 

Completion 

Month 

Major  Task  1:  Obtain  local  IRB  and  VA  R&D  and  HRPO 
Approvals. 

1-2 

1 

Subtask  1:  File  protocol  with  Local  IRB 

1 

1 

Subtask  2:  File  protocol  with  VA  R&D 

1 

1 

Subtask  3:  File  protocol  with  HRPO 

1-2 

1 

Subtask  4:  Make  any  required  revisions  and  resubmit  in  the 
above  order. 

1-2 

3 

Subtask  5:  Obtain  Local  IRB/ VA  R&D/  HRPO  Approval 

2 

1 

Milestone  #1 :  Local  IRB/  VA  R&D/  HRPO  Approval 

1 

Aim  1 :  To  conduct  a  developmental  formative  evaluation  to  Iteratively  Inform  a  Virtual 

World  (VW)  design  based  on  our  prior  web-based  posttraumatic  stress  (PTS)  training  for 
primary  care  providers  (PCPs). 

5 


Major  Task  2:  Semi-Structured  interviews  with  project 
stakeholders/key  informants  to  inform  curriculum  content 
and  instructional  design 

3 

7-10 

Subtask  1:  Recruit  &  enroll  stakeholders/key  informants 

3 

7-9 

Subtask  2:  Qualitative  analysis  of  curriculum  content  and 

O 

9-12 

instructional  design. 

o 

Major  Task  3:  Begin  Virtual  World  (VW)  build 

2-12 

7-12 

Subtask  1:  Design  curriculum  based  on  data  from  semi-structured 
interviews. 

2-4 

8-12 

Subtask  2:  Host  VW  learning  environment  (initially  in  Second  Life) 

5 

7 

Subtask  3:  Build  an  Orientation  Center 

6 

7 

Subtask  4:  Create  a  storyboard 

7 

9 

Subtask  5:  Import  and  create  virtual  objects 

9 

9 

Subtask  6:  Create  avatar  types 

9 

9 

Subtask  7:  Secure  the  VW  environment 

10 

incomplete 

Subtask  8:  Conduct  quality  checks 

11 

incomplete 

Subtask  9:  Migrate  to  other  VW  platform  (e.g.  UNITY) 

12 

incomplete 

What  was  accomplished  under  these  goals? 

Major  Task  1:  Obtain  local  IRB  and  VA  R&D  and  HRPO  Approvals.  -  Complete 
Subtask  1:  File  protocol  with  Local  IRB 
Subtask  2:  File  protocol  with  VA  R&D 
Subtask  3:  File  protocol  with  HRPO 

Subtask  4:  Make  any  required  revisions  and  resubmit  in  the  above  order. 

Subtask  5:  Obtain  Local  IRB/  VA  R&D/  HRPO  Approval 

Submitted  to  and  Approved  by: 

University  of  Caiifornia  San  Francisco  Committee  on  Human  Research- 14-15004 

Study  Approval  (Continuing  Review)  09/03/2016-05/04/2017 
Amendment  1  (personnel  changes)-  08/12/2015 
Amendment  2  (personnel  changes)-  21/12/2015 

Amendment  3  (personnel  changes,  increased  number  of  SSIVs,  interview  guide)- 
09/03/2016 

VA  Research  &  Development  and  Ciinicai  Research  Workgroup- 

CRW-  Approval  22/04/2015 
VA  R&D  -  Approval  07/05/2015 
-  Renewed  09/03/2016 

DoD  Human  Research  Protection  Office  A18590 

Approval  17/9/2015 

Continuing  Review  Approved  26/04/2016 
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Major  Task  2:  Semi-Structured  interviews  with  project  stakeholders/key 
informants  to  inform  curriculum  content  and  instructional  design  -  Complete 

The  qualitative  team  interviewed  1 1  PCP  “end  users”  and  healthcare  leadership. 
Additional  interviews  will  be  conducted  with  educational  experts  and  Information 
Technology  (IT)  experts  in  the  first  half  of  Year  2.  The  Qualitative  Researchers  assert 
that  this  will  produce  the  most  relevant  data,  since  there  will  be  beta  elements  of  the 
training  to  share  and  discuss  with  interviewees  at  different  milestones  of  the  curriculum 
and  virtual  training  development. 

The  overall  conclusions  from  the  developmental  formative  evaluation  include: 

1 .  Findings  about  challenges  in  primary  care  practice  related  to  providing  care  to 
veterans  with  PTSD  as  well  as  treatment  choices  by  PCPs  and  use  of  motivational 
interviewing  are  consistent  with  the  published  studies. 

2.  Findings  support  the  need  for  training  for  PCPs  focused  on  PTSD  and  Ml  applied  to 
providing  care  for  patients  with  PTSD. 

3.  Opinions  of  interviewed  PCPs  about  VW  training  were  limited  to  their  insufficient 
understanding  of  VW. 

4.  This  formative  evaluation  generated  implications  for  audience  generation,  content, 
and  training  delivery  and  evaluation,  and  training  generalizability  that  are  being 
utilized  for  the  VW  training  development  in  this  project. 

Subtask  1:  Recruit  &  enroll  stakeholders/key  informants 

•  Project  Coordinator  and  Research  Assistant  recruited  and  enrolled 
stakeholders/key  informants  from  a  potential  pool  of  participants  established  by 
the  PI  and  Project  Team. 

•  Recruitment  began  on  19-04-2016  and  will  continue  as  deemed  necessary. 

•  PI  and  Qualitative  Researcher  have  conducted  11  semi-structured  interviews,  to 
date. 

Subtask  2:  Qualitative  analysis  of  curriculum  content  and  instructional 
design. 

•  Qualitative  Analyst  analyzed  semi-structured  interview  results  in  relation  to  the 
curriculum  content  and  instructional  design. 

•  Content  experts  from  the  project  team  have  reviewed  and  offered  insight  on 
curriculum  storyboards. 

•  Findings  from  the  analyses  often  interviews  are  shared  in  a  Technical  Report 
(Appendix  A). 

Major  Task  3:  Begin  Virtual  World  (VW)  build  -  In  Progress  -  70% 

Subtask  1:  Design  curriculum  based  on  data  from  semi-structured 
interviews. 

•  Analyzed  evaluation  data  was  displayed  in  a  matrix  (within  Appendix  A),  and 
shared  with  the  curriculum  development  team  to  inform  the  refinement  of  the 
training  curriculum. 
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•  Storyboards  detailing  the  content  for  Sessions  1  and  2  of  the  training  were 
refined  and  submitted  to  CNDG.  CNDG  will  transform  the  concepts  into  a  beta 
version  of  the  training. 

Subtask  2:  Host  VW  learning  environment  (initially  in  Second  Life) 

•  Project  Team  members  have  been  oriented  to  Second  Life.  CNDG  hosted  team 
members  in  an  existing  learning  environment  and  solicited  feedback  regarding 
the  characteristics  of  virtual  spaces  and  avatars  to  help  inform  the  environment 
they  will  build  specifically  for  this  project  during  a  project-wide  meeting  on  26-04- 
2016. 

•  CNDG  submitted  a  video  detailing  their  progress  on  22-06-2016. 

•  CNDG  created  a  virtual  campus  (Figures  1  and  2,  Appendix  B),  which  contains 
a  hospital  building  with  internal  exam  rooms  for  Standardized  Patient  interviews 
(Figure  7,  Appendix  B). 

•  As  the  developers  at  CNDG  build  additional  training  components,  the  Project 
Team  will  continue  to  monitor  progress  and  offer  feedback. 

Subtask  3:  Build  an  Orientation  Center 

•  CNDG  held  several  orientation  sessions  for  project  staff  and  collaborators  in  the 
virtual  world.  The  Orientation  Center  environment  provides  instructions  on  how  to 
navigate,  as  well  as  serves  as  a  test  for  computer  compatibility  (Figures  3  and  4, 
Appendix  B). 

Subtask  4:  Create  a  storyboard 

•  Heyden  Ty,  our  vendor  which  specializes  in  virtual  learning  curriculums, 
submitted  storyboards  for  each  segment  of  the  two-session  training  on  15-06- 
2016.  These  storyboards  contain  the  proposed  learner  experience  during  the 
various  segments  of  the  training:  Exploratorium,  Didactic/Lecture,  Small  Group 
Standardized  Patient  Interviews,  and  Homework.  Additionally,  they  contain 
content  and  directions  for  the  engineering  experts  at  CNDG. 

•  Much  of  Quarter  4  was  spent  refining  these  storyboards  to  include  feedback 
from  the  developmental  formative  evaluation. 

•  Final  versions  of  the  storyboards  were  submitted  by  Heyden  Ty  on  23-09-2016 
(Appendices  C,  D,  E). 

Subtask  5:  Import  and  create  virtual  objects 

•  CNDG  created  a  video  which  presents  virtualized  elements  of  the  storyboards 
and  curriculum,  which  are  currently  hosted  on  Second  Life  for  the  project  team 
and  Principal  Investigator  to  review.  Examples  include  polling  stations, 
presentation  screens,  whiteboards  for  instructor  and  learner  (Figure  5,  Appendix 
B),  and  click-to-view  information  posters.  They  will  continue  to  develop  virtual 
objects  and  refine  the  environment  in  Quarter  4,  as  the  storyboards  are  finalized. 

Subtask  6:  Create  avatar  types 

•  CNDG  created  sample  avatars  for  the  training,  based  on  the  specifications 
provided  to  them  by  the  Project  Coordinator.  Additionally,  they  created  a 
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prototype  of  the  veteran,  which  will  serve  as  the  case  study  for  the  duration  of  the 
training  (Figures  6  and  7,  Appendix  B).  The  project  team  will  review  the  avatars 
and  provide  feedback  for  modifications  during  Quarter  4. 

Subtask  7:  Secure  the  VW  environment 

Subtask  8:  Conduct  quality  checks 

Subtask  9:  Migrate  to  other  VW  platform  (e.g.  UNITY) 

Subtasks  7-9  are  incomplete,  because  the  virtual  world  environment  and  its 
associated  elements  and  avatars  are  still  in  the  design  phase.  The  delay  in 
implementing  the  semi-structured  interviews  delayed  the  design  process.  These 
subtasks  will  be  completed  in  early  Year  2  in  order  to  evaluate  the  design  and 
content,  and  to  then  proceed  with  training  delivery. 

What  opportunities  for  training  and  professional  development  has  the  project 
provided? 


Nothing  to  report. 


How  were  the  results  disseminated  to  communities  of  interest? 


Nothing  to  report. 


What  do  you  plan  to  do  during  the  next  reporting  period  to  accomplish  the  goals? 


CNDG  will  formally  develop  the  virtual  elements  and  environment  according  to  the 
storyboards  and  curriculum  provided  by  the  project  staff  and  Heyden  Ty.The  research 
team  will  then  move  forward  with  the  SOW  goals,  including  the  independent  review  and 
implementation  of  the  virtual  world  training. 

Major  Task  4:  Independent  review  of  new  VW  training  using  a  focus  group 
Major  Task  5:  Refinement  of  the  VW  training 

Major  Task  6:  Refinement  of  prior  online  training  (Control)  to  make  it  a  more  apt 
comparison  for  RCT. 
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4.  IMPACT 


What  was  the  impact  on  the  development  of  the  principal  discipline(s)  of  the 
project? 


The  qualitative  data  collected  as  part  of  the  developmental  formative  evaluation  was 
summarized  in  a  technical  report,  which  the  researchers  plan  to  develop  into  a  manuscript. 
When  the  manuscript  is  accepted  for  publication  in  a  peer-reviewed  journal,  it  will  inform 
the  field  of  qualitative  evaluation. 

While  it  is  too  early  to  cite  specifics,  the  project  also  has  the  potential  to  make  an  impact  on 
primary  care  provider  education  and  continuing  medical  education  relating  to  health  issues 
affecting  both  mind  and  body,  including  post-traumatic  stress  disorder.  Since  the  training 
is  delivered  virtually,  it  can  be  disseminated  world-wide. 


What  was  the  impact  on  other  disciplines? 

Nothing  to  report. 


What  was  the  impact  on  technology  transfer? 


Nothing  to  report. 


What  was  the  impact  on  society  beyond  science  and  technology? 


The  developmental  formative  evaluation  revealed  a  fair  amount  of  hesitation  toward  virtual 
world  training  for  the  sake  of  continuing  medical  education.  Several  PCPs  indicated 
generational  differences  in  acceptability  and  ability  to  navigate  and  learn  in  a  virtual 
environment.  As  the  findings  are  applied  to  the  curriculum  and  design  of  the  training,  the 
project  has  the  potential  to  improve  public  knowledge  and  attitudes  toward  virtual  world 
technology  for  the  sake  of  provider  education. 
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5.  CHANGES/PROBLEMS 


Changes  in  approach  and  reasons  for  change 

Nothing  to  report. 


Actual  or  anticipated  problems  or  delays  and  actions  or  plans  to  resolve  them 

1.  Resolved:  Although  the  project  year  began  on  25  September  2015,  the  Project 
Coordinator  was  not  hired  until  16  November,  creating  minor  delays  with  some 
project  activities. 

2.  Resolved:  Establishing  vendors  delayed  build  phase.  Vendors  were  provided 
with  funds  in  January  and  moved  forward  with  work  on  the  project. 

3.  Delay:  The  project  team  members  responsible  for  collecting  and  analyzing  the  qualitative 
data  via  semi-structured  interviews  had  taken  longer  than  expected  to  begin  recruitment. 
Additional  time  was  necessary  to  formalize  the  interview  guides,  and  then  develop  a  large 
enough  sample  for  the  PCP  interviews. 

•  Resolution:  A  total  of  1 1  interviews  were  completed.  Qualitative  researchers  will  likely 
conduct  additional  interviews  as  necessary. 

4.  Delay:  As  previously  stated.  Subtasks  7-9  associated  with  the  virtual  world  build  were 
delayed  as  a  result  of  the  modified  timeline  for  the  implementation  and  analysis  of  the  semi- 
structured  interviews.  This  work  will  be  completed  in  early  Year  2  of  the  project. 

5.  Ongoing  Problem:  Second  Life,  the  online  platform  which  will  host  the  training  during  the 
development  phase,  is  blocked  on  the  VA  network.  This  will  impede  on  several  research 
components,  including  delivery  of  the  virtual  world  (experimental)  version  of  the  training,  if  it  is 
not  resolved. 

•  Course  of  action  being  taken:  Project  Staff  and  CNDG  are  investigating  alternative 
options  for  delivery  of  the  training  during  the  focus  group  and  RCT  phases  of  the  study. 


Changes  that  had  a  significant  impact  on  expenditures 

Less  was  expended  on  payroll  than  in  the  initial  budget.  The  Project  Coordinator  not  being 
hired  until  well  into  Month  2  of  the  project,  accounting  for  some  of  the  disparity.  Additionally, 
the  salary  support  of  certain  personnel  was  less  than  anticipated  in  Year  1 ,  but  will  increase  in 
Year  2. 

The  project’s  travel  budget  reflected  an  annual  trip  to  present  project  findings  in  Washington, 
DC.  Since  such  a  meeting  did  not  occur,  the  travel  budget  was  underutilized  during  Year  1 . 
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Significant  changes  in  use  or  care  of  human  subjects,  vertebrate  animais, 
biohazards,  and/or  seiect  agents 

Nothing  to  report. 


Significant  changes  in  use  or  care  of  human  subjects 


Nothing  to  report. 


Significant  changes  in  use  or  care  of  vertebrate  animais. 


Not  applicable. 


Significant  changes  in  use  of  biohazards  and/or  seiect  agents 


Not  applicable. 
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6.  PRODUCTS 


An  abstract  was  submitted  for  consideration  to  present  findings  from  the 
developmental  formative  evaluation  at  a  conference  in  2017.  The  authors  will  be 
informed  of  its  status  early  in  Project  Year  2. 

•  litje  -  USING  RAPID  QUALITATIVE  ANALYSIS  TO  SUPPORT  THE 
DEVELOPMENT  AND  IMPLEMENTATION  OF  A  VIRTUAL  WORLD  TRAINING 
FOR  PRIMARY  CARE  PROVIDERS  ON  CARING  FOR  VETERANS  WITH 
POST-TRAUMATIC  STRESS  DISORDER  SYMPTOMS 

•  Authors  -  Marianna  B.  Shershneva,  MD,  PhD;  Christopher  J.  Koenig,  PhD; 
Mathew  Douraghi,  MA;  Eilleen  E.  Sabino,  MPH;  Karen  H.  Seal,  MD,  MPH 

•  Submitted  for  consideration  to:  2017  Society  for  Academic  Continuing  Medical 
Education’s  Annual  Meeting  in  May  2017  as  Research  in  Continuing  Medical 
Education  (RICME),  Works  in  Progress 


Books  or  other  non*periodical,  one-time  publications. 

Nothing  to  Report. 


Other  publications,  conference  papers,  and  presentations. 


The  qualitative  team  produced  a  Technical  Report  (Appendix  A)  for  internal  use. 
The  report  details  data  collected  through  semi-structured  interviews  with  PCPs 
and  its  relevance  to  current  research  in  the  field  of  PTSD  treatment,  as  well  as 
virtual  world  education.  The  intention  is  to  further  develop  the  report  into  a 
manuscript,  then  submit  it  to  a  peer-reviewed  journal  in  the  coming  project  year. 


Website(s)  or  other  Internet  site(s) 

A  project  website  focused  on  recruitment  and  dissemination  of  project  information 
will  be  developed  and  launched  during  Project  Year  2. 
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Technologies  or  techniques 


Nothing  to  report. 


Inventions,  patent  appiications,  and/or  iicenses 


Nothing  to  report. 


Other  Products 


There  are  several  products  for  internal  use,  which  facilitate  the  design  and 
implementation  of  the  virtual  world  training: 

•  A  storyboard  representing  the  overall  look  and  feel  of  the  virtual  world 
environment.  (Appendix  C) 

•  A  storyboard  for  each  of  the  two  training  sessions.  (Appendices  D  and  E) 
o  Within  these  storyboards  is  the  framework  for  “Exploratorium”  learning 

stations  that  potentially  can  be  used  as  stand-alone  training  tools. 

•  A  video  produced  by  CNDG,  which  showcases  the  virtual  elements  they  have 
created  to  date. 

o  Due  to  file  size,  it  can  only  be  shared  via  Dropbox  or  another  file  sharing 
application.  Still  photos  of  the  video  can  be  found  in  Appendix  B. 
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7.  PARTICIPANTS  &  OTHER  COLLABORATING  ORGANIZATIONS 


What  individuals  have  worked  on  the  project? 


Name:  Karen  Seal,  MD,  MPH 

Project  Role:  Principal  Investigator 

Nearest  person  month  worked:  2.4 
Contribution  to  Project:  Unchanged 

Name:  Eilleen  Sabino-Laughlin 

Project  Role:  Project  Coordinator 

Nearest  person  month  worked:  12 
Contribution  to  Project:  Unchanged 

Name:  Nicole  R.  McCamish 

Project  Role:  Project  Manager 

Nearest  person  month  worked:  7.2 
Contribution  to  Project:  Unchanged 

Name:  Linda  Abadjian,  PhD 

Project  Role:  Evaluator 

Nearest  person  month  worked:  1 .8 
Contribution  to  Project:  Unchanged 

Name:  Yongmei  Li,  PhD 

Project  Role:  Statistician 

Nearest  person  month  worked:  6 
Contribution  to  Project:  Unchanged 

Name:  Tom  Metzler,  MS 

Project  Role:  Data  Manager 

Nearest  person  month  worked:  1 .2 
Contribution  to  Project:  Unchanged 

Name:  Shira  Maguen,  PhD 

Project  Role:  Co-Investigator 

Nearest  person  month  worked:  0.6 
Contribution  to  Project:  Unchanged 

Name:  Thomas  Neylan,  MD 

Project  Role:  Co-Investigator 

Nearest  person  month  worked:  0.3 
Contribution  to  Project:  Unchanged 


Name:  Beth  Cohen,  MD,  MAS 

Project  Role:  Co-Investigator 

Nearest  person  month  worked:  0.6 
Contribution  to  Project:  Unchanged 

Name:  Greg  Reger,  PhD 

Project  Role:  Co-Investigator 

Nearest  person  month  worked:  0.6 
Contribution  to  Project:  Unchanged 

Name:  Christopher  Koenig,  PhD 
Project  Role:  Co-Investigator 

Nearest  person  month  worked:  0.6 
Contribution  to  Project:  Unchanged 

through  the  end  of  July,  when 
Dr.  Koenig  left  his  appointment 
at  the  San  Francisco  VA. 

Vendors 

Name:  Chant  Newall  Development  Group 
Project  Role:  Consultant 
Paid  by  deliverables  per  the  SOW 
Contribution  to  Project:  Unchanged 

Name:  Forefront  Collaborative 
Project  Role:  Consultant 
Paid  hourly  on  a  semi-annual  basis 
Contribution  to  Project:  Unchanged 

Name:  Heyden  Ty 

Project  Role:  Consultant 

Paid  by  deliverables  per  the  SOW 

Contribution  to  Project:  Unchanged 
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Has  there  been  a  change  in  the  active  other  support  of  the  PD/PI(s)  or  senior/key 
personnel  since  the  last  reporting  period? 


There  are  no  changes  in  active  other  support  (for  the  PI  and  Co-I’s)  that  would  significantly 
impact  the  effort  on  this  project. 


What  other  organizations  were  involved  as  partners? 


Nothing  to  report. 


8.  SPECIAL  REPORTING  REQUIREMENTS 
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Gantt  Chart  W81XWH-15-C-0088 


Improving  Access  to  Care  for  Warfighters:  Virtual 
Worlds  Technology  to  Enhance  Primary  Care  Training  I 

1 

FY2016 

FY  2017 

FY  2018 

FY  2019 

Q2 

Q3 

Q4 

Q1 

Q2 

Q3 

Q4 

Q1 

Q2 

Q3 

Q4 

Q1 

Q2 

Q3 

Q4 

Major  Task  2:  Semi-Structured  interviews 
with  project  stakeholders/key  informants  to 

Anticipated 

N=up 
to  20 

inform  curriculum  content  and  instructional 
design 

Actual 

N= 

11 

Aim  1 

Formative 

Eval/Design 

Major  Task  4:  Independent  review  of  new 
VW  training  using  a  focus  group  -  Recruit 

Anticipated 

N=8 

stakeholders/key  informants  who  were  not 
involved  in  initial  build  recommendations 

Actual 

Major  Task  5:  Refinement  of  the  VW 
training  -  Independent  reviewers  for  Focus 
Group 

Anticipated 

N=8 

Actual 

Aim  2  RCT 

Major  Task  1 1 :  Recruit  PCPs  from  VA, 

Anticipated 

N=100 

DoD,  and  community 

Actual 

Major  Task  19:  Conduct  Summative 
Analysis  -  Recruit  PCPs  from  each  training 

arm 

Anticipated 

N= 

20 

Aim  3 
Summative 

Actual 

Eval/ 

Disseminate 

Major  Task  19:  Conduct  Summative 

Anticipated 

N= 

:10 

Analysis  -  Recruit  stakeholders  from  Aim  1 

Actual 

Updated  10/21/2016 


“Improving  Access  to  Care  for  Warfighters:  Virtual  Worlds  Technology  to  Enhance  Primary 
Care  Training  in  Posttraumatic  Stress  and  Motivational  Interviewing” 

Log  #JW140067  Award  #  W81XWH-15-C-0088 


PI:  Karen  Hope  Seal,  MD  MPH  Org:  Northern  California  Institute  for  Research  and  Education  Award  Amount:  $1,554,345.30  Directs 


Study  Aims 

Veterans  present  to  primary  care  providers  (PCPs)  with  posttraumatic  stress  (PTS)  symptoms  because  many 
are  resistant  to  specialty  mental  health  care.  Most  PCPs  have  not  been  trained  to  assess  for  and  manage 
symptoms  of  PTS  or  motivate  Veterans  to  engage  in  treatment.  This  can  result  in  missed  opportunities  to 
intervene  to  prevent  chronic  mental  and  physical  health  problems.  We  propose  to: 

(1)  Iteratively  design  a  new  web-based  PTS  and  Motivational  Interviewing  training  for 
PCPs  using  Virtual  World  technology  to  enhance  interactivity. 

(2)  Add  a  more  robust  evaiuation  inciuding  a  randomized  controi  trial  for  more  clinicaliy 
valid  outcome  measurement. 

(3)  Conduct  a  summative  evaluation  to  inform  national  “scale-up”  dissemination  and 
implementation. 

The  proposed  project  is  aligned  with  the  needs  of  JPC  5  (Psychological  Health  and  Resilience)  and  will 
produce  a  deliverable  that  will  improve  access  to  quality  clinical  care  for  our  warfighters  suffering  with  PTS. 

Approach 

We  are  using  mixed  qualitative  and  quantitative  observational  and  experimental  methods  to  conduct  a 

4-vear  effectiveness-implementation  randomized  controlled  trial  (RCT).  in  which  project  stakeholders 

participate  from  start  to  finish.  A  formative  evaluation  consisting  of  focus  groups  and  semi-structured 
interviews  captures  stakeholder  input  in  how  we  can  best  design  and  implement  the  Virtual  World  (VW) 
training.  We  will  then  conduct  an  RCT  to  compare  the  new  VW  training  to  our  prior  online  PTS  training.  Pre- 
/post-  and  follow-up  standardized  patient  interviews,  provider  self-report  measures,  and  patient  outcomes  will 
be  compared  between  groups.  A  summative  evaluation  will  solicit  feedback  of  PCP  participants  and 
stakeholders  to  expedite  dissemination  of  the  new  VW  training. 


Accomplishment:  Beta  versions  of  the  learning  environment, 
Orientation  Center,  avatar  types,  virtual  objects,  and  key 
exercises  of  the  training,  as  described  in  the  storyboards. 


Timeline  and  Cost 


Goals/Milestones 


Projected  Period  of  Performance  and  Proposed  Direct  Costs  Per  Year 


RCT 

Aim  3  Summative 
Eval/Disseminate 

Estimated  Costs/Year  495,224.05  436,314.56  310,165.91  312,640.78 

($) 

Total  Direct:  $1,554,345.30 

Period  of  Performance:  01/10/2015-30/09/2016 
Updated:  24/10/2016 


Major  Task  1 :  Obtain  iocal  IRB  and  VA  R&D  and  HRPO  Approvals 
Major  Task  2:  Semi-Structured  interviews  with  project  stakeholders/key 
informants  to  inform  curriculum  content  and  instructional  design 
Major  Task  3:  Begin  Virtual  World  (VW)  build 

Subtask  1 :  Design  curriculum  based  on  data  from  semi-structured  interviews 
Subtask  2:  Host  VW  learning  environment  (initially  in  Second  Life) 

Subtask  3:  Build  an  Orientation  Center 
Subtask  4:  Create  a  storyboard 
Subtask  5:  Import  and  create  virtual  objects 
Subtask  6:  Create  avatar  types 

Budget  Expenditure  to  Date 
Projected  Expenditure:  $495,224.05 
Actual  Expenditure:  $399,346.00 


9.  APPENDICES 
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Appendix  A:  Technical  Report  for  Developmental  Formative  Evaluation 

Interviews  with  Primary  Care  Providers  to  Inform  the  Development  of  the  Virtual  World 
Training  on  Posttraumatic  Stress  and  Motivational  Interviewing 

TECHNICAL  REPORT 
23  October  2016 

Unpublished  Data  -  Not  for  Distribution 
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References . 13 
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Background 


Military  veterans  with  posttraumatic  stress  disorder  (PTSD)  symptoms  receive  care  from  primary  care  providers 
(PCPs)  who  often  lack  basic  skills  in  detecting  and  managing  PTSD  and  in  using  effective  communication 
techniques.  Improving  Access  to  Care  for  Warfighters:  Virtual  Worlds  Technology  to  Enhance  Primary 
Care  Training  in  Posttraumatic  Stress  and  Motivational  Interviewing  is  a  four-year  project  with  the 
overarching  goal  to  build  competency  among  a  primary  care  workforce  to  better  detect  and  manage 
posttraumatic  stress  symptoms  and  motivate  treatment  engagement  in  warfighters  through  the  use  of  virtual 
world  (VW)  technology.  This  project  consists  of  development  of  an  innovative  VW  training  for  primary  care 
providers  (PCPs)  and  then  conducting  a  randomized  controlled  trial  (RCT)  to  compare  its  effectiveness  against 
a  traditional  web-based  course  covering  similar  content.  This  project  is  funded  as  a  Joint  Warfighter  initiative^  by 
the  US  Department  of  Defense. 

This  report  presents  the  preliminary  results  from  a  developmental  formative  evaluation,  which  is  part  of  Phase  1 
of  the  project.  The  qualitative  research  team  sought  to  inform  the  design  of  the  aforementioned  VW  training 
through  semi-structured  interviews  with  key  stakeholders,  particularly  PCPs,  who  are  the  target  audience  for  the 
training.  Contributing  researchers  include:  Christopher  J.  Koenig,  PhD;  Marianna  Shershneva,  PhD;  Eilleen 
Sabino-Laughlin,  MPH;  and  Mathew  Douraghi,  MA  under  the  guidance  of  the  Principal  Investigator,  Karen  H. 
Seal,  MD  MPH.  Research  activities  were  approved  through  the  University  of  California  San  Francisco 
Committee  on  Human  Research,^  VA  Research  and  Development  Clinical  Research  Workgroup,  and  the 
Department  of  Defense  Human  Research  Protection  Office^. 


Purpose 

This  developmental  formative  evaluation  study  employs  qualitative  methods  to  explore  perspectives  of  PCPs, 
educators,  health  care  leadership,  and  information  technology  specialists  on  the  relevance,  acceptability,  and 
feasibility  of  the  VW  training.  This  evaluation  is  being  conducted  to  solicit  input  from  these  stakeholder  groups  to 
help  shape  the  intervention  content  and  execution  in  the  VW  environment.  The  research  team  focused  on  PCP 
stakeholders  first  because  understanding  their  experience  with  and  perspectives  on  PTSD  assessment  and 
management  is  critical  to  planning  and  implementing  the  VW  training  modules. 


Methods 

study  Design 

The  developmental  formative  evaluation  has  been  qualitative  in  nature.  Data  have  been  collected  using  semi- 
structured  interviews  and  analyzed  using  rapid  qualitative  analysis,  a  collaborative  process  involving 
triangulation,  iterative  data  collection  and  analysis  procedures  to  quickly  develop  an  understanding  of  a  target 
area  from  stakeholders’  perspectives.  Emerging  themes  from  the  analysis  are  being  used  by  the  VW 
instructional  design  experts  to  tailor  the  content  and  its  presentation  to  the  needs,  values,  and  preferences  of 
the  stakeholders.  This  process  should  facilitate  future  implementation  and  dissemination  among  the  project 
stakeholders. 

Sample  and  Recruitment 

A  convenience  sample  of  PCPs  affiliated  with  the  Veterans  Health  Administration  (VHA)  was  recruited  via  e- 
mail.  The  e-mail  included  the  study  name,  a  brief  “Dear  Colleague”  message  from  the  study  Principal 
Investigator,  Dr.  Karen  Seal,  inviting  VA  PCPs  to  participate,  and  a  detailed  information  sheet  describing  the 
study  purpose,  interview  length,  anticipated  risks  and  benefits,  privacy  and  confidentiality  notices,  and 
participation  compensation.  Each  e-mail  solicitation  included  a  link  to  a  YouTube  video.''  This  video  was  created 


'  Award  #  W81XWH-15-C-0088,  Principal  Investigator  Karen  Hope  Seal,  MD  MPH 
"  CHR  14-15004,  Interview  Guide  Approved:  09/03/2016 
"HRPO  A18590 

''  VW  video:https://www. youtube. com/watch?v=N  WaUT77LwU&feature=voutu.be 
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by  the  VW  consultant  team  to  provide  an  example  of  a  multi-media,  immersive  VW  training  environment  and  to 
introduce  prospective  participants,  who  might  have  limited  experience  with  VW,  to  the  use  of  the  VW  technology 
for  training  and  education.  Interested  PCPs  were  asked  to  respond  to  the  e-mail  solicitation  and  were  scheduled 
for  an  interview  by  the  qualitative  team. 

Data  Collection 

The  research  team  developed  an  original  semi-structured  interview  guide  based  on  the  senior  authors’  clinical 
experience  and  Drs.  Koenig’s  and  Shershneva’s  experience  conducting  qualitative  research  to  develop 
education  interventions.  The  PCP  interview  guide  included  questions  designed  to  collect  data  with  respect  to 
four  domains:  (1 )  the  applicability  of  training  topics  for  PCPs,  including  PTSD-related  content  and  Ml  training;  (2) 
the  feasibility  and  acceptability  of  a  synchronous  VW  training  for  busy  PCPs;  (3)  barriers  to  and  facilitators  of 
implementation  with  regard  to  learner  burden  vs.  degree  of  interactivity,  internet  access,  bandwidth  and  security 
issues,  and  preferences  for  asynchronous  training  exercises  and  downloadable  provider  and  patient  educational 
materials;  and  lastly  (4)  generalizability  for  implementation  among  PCPs  in  diverse  practice  settings  with 
different  populations. 

Ten  semi-structured  interviews  with  PCPs,  including  several  clinicians  who  also  had  leadership  positions  in  their 
organizations,  were  conducted  by  experienced  interviewers.  Dr.  Christopher  J.  Koenig  and  Mathew  Douraghi 
from  May  to  July,  2016.  All  interviews  except  one  were  conducted  on  the  phone  and  were  approximately  30 
minutes  long.  One  interview  was  conducted  in-person  by  the  interviewee’s  request.  All  participants  agreed  to 
have  their  interviews  digitally  audio  recorded.  Data  included  the  audio  recordings  and  interviewer  notes  taken 
during  the  interview.  The  recordings  were  retained  and  used  to  verify  hand-written  notes  and  to  identify 
particularly  rich  or  meaningful  participant  responses,  some  of  which  were  selected  and  incorporated  in  the  rapid 
analysis. 

Interviews  with  educators  and  IT  specialists  will  continue  throughout  intervention  development.  In  the  course  of 
the  interview,  several  participants  volunteered  their  expertise  and  time  to  examine  and  evaluate  prototypes  of 
the  training  modules  and  training  intervention. 

Data  Analysis 

The  first  step  of  the  rapid  analysis  involved  summarizing  interviewer  notes  using  a  structured  template  that 
maps  onto  the  interview  guide  topics.  For  instance,  immediately  after  the  interview,  Mr.  Douraghi  listened  to  the 
audio  recording,  reviewed  notes  taken  during  the  interview,  summarized  the  notes  into  the  template,  and 
included  verbatim  quotations  that  illustrated  particularly  rich  or  meaningful  content.  Subsequently,  Dr.  Koenig 
reviewed  the  summary  and  added  his  notes  and  comments.  Additionally,  Mr.  Douraghi  added  the  interview 
recording  timestamps  next  to  the  key  statements  to  facilitate  quick  retrieval  of  the  corresponding  segment  of  the 
recording,  if  additional  data  review  were  needed.  Completed  summaries  were  collaboratively  reviewed  by  Drs. 
Koenig  and  Shershneva  and  Mr.  Douraghi  for  accuracy  and  relevance  to  the  four  domains  (i.e.,  relevance, 
feasibility/acceptability,  barriers  and  facilitators,  and  generalizability),  and  by  Ms.  Sabino-Laughlin  (Project 
Manager)  for  relevance  to  intervention  development. 

The  second  step  involved  transferring  individual  interview  content  from  the  original  template  onto  a  matrix. 

Matrix  displays  are  a  common  rapid  qualitative  analysis  technique  to  further  summarize  interview  content  to 
identify  similarities  and  differences  across  participant  responses.  Particularly  rich  content  was  noted  on  the 
display  to  retain  participants’  concerns  in  their  voices.  The  evolving  matrix  display  was  discussed  by  the 
qualitative  research  team  to  compare  findings  across  participant  groups  and  identify  implications  for  the  VW 
training  content  and  delivery.  Finally,  Dr.  Shershneva,  Mr.  Douraghi,  and  Ms.  Sabino-Laughlin  presented  the 
qualitative  findings  in  the  form  of  the  matrix  display  to  the  intervention  development  team  for  discussion  of 
themes  and  implications.  Several  meetings  among  the  members  of  the  qualitative  research  team  and  the 
curriculum  development  team  resulted  in  gaining  insights  into  the  PCPs’  perspectives  that  helped  affirm  or 
modify  how  the  VW  intervention  modules  might  be  refined  to  be  responsive  to  stakeholder  concerns. 
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Results 


Participants 

All  study  participants  were  PCPs.  They  ranged  In  age  and  experience,  and  were  located  In  four  different  states, 
with  seven  located  In  California,  one  In  Connecticut,  one  In  Colorado,  and  one  In  Minnesota.  All  California  PCPs 
were  from  the  San  Francisco  Bay  Area.  Half  of  the  participants  were  females;  70%  (n=7)  were  physicians  and 
the  remaining  30%  (n=3),  nurse  practitioners.  All  male  participants  were  physicians.  The  female  participants 
Included  three  nurse  practitioners  and  two  physicians.  All  were  affiliated  with  the  VA  system,  and  all  had  no  or 
limited  experience  using  VW. 


Profession 

Gender 

Age  Category 

Location 

VW  Experience 

Physician 

Male 

40-49 

California 

None 

Physician 

Male 

50-59 

California 

None 

Physician 

Male 

60-69 

California 

Limited,  not  similar  to  this 
training 

Physician 

Female 

40-49 

California 

None 

Physician 

Male 

30-39 

Colorado 

Limited,  not  similar  to  this 
training 

Physician 

Male 

40-49 

Minneapolis 

None 

Physician 

Female 

60-69 

Connecticut 

None 

Nurse  Practitioner 

Female 

50-69 

California 

Limited,  not  similar  to  this 
training 

Nurse  Practitioner 

Female 

30-39 

California 

None 

Nurse  Practitioner 

Female 

50-59 

California 

Limited,  not  similar  to  this 
training 

Relevance  of  Training  Topic  and  Curriculum 
Difficulties  in  Recognizing  and  Diagnosing  PTSD 

Half  of  the  Interviewees  Identified  that  PTSD  manifests  In  different  ways  and  reported  the  challenge  of 
differentiating  PTSD  from  substance  abuse,  anxiety,  alcoholism,  depression,  and  other  mental  health  conditions. 
One  PCP  acknowledged  difficulty  distinguishing  actual  trauma  from  imagined  trauma  due  to  dementia  or 
psychosis,  and  another  PCP  commended  on  the  difficulty  in  differentiating  PTSD  from  other  mental  health 
conditions,  especially  in  older  patients.  Below  are  examples  from  two  interviews: 

•  “The  main  challenge  is  that  most  of  us  in  primary  care,  we’re  not  mental  health  practitioners.  We’re  not 
as  experienced  with  mental  health,  it’s  an  issue  with  training,  exposure,  and  experience.  And  also  a  little 
bit,  maybe  for  some,  sometimes  if  you  don’t  have  enough  experience,  you’re  not  comfortable.  That 
would  be  a  main  challenge.”  (P6) 

•  “It  [PTSD]  can  feel  a  little  occult  sometimes,  kind  of  hidden  behind  something  that  looks  more  like 
substance  abuse  or  generalized  anxiety  or  alcoholism  or  sometimes  I  just  won’t  get  answers  during  my 
interview.”  (P2) 

Several  PCPs  reflected  on  the  limitations  of  available  screening  tools  for  PTSD,  for  example  saying  that  four 
screening  questions  may  not  be  enough  to  identify  PTSD  and  that  there  may  be  false-negative  screening 
results. 

Two  PCPs  elaborated  on  difficulties  determining  the  etiology  of  PTSD  in  veterans,  which  may  be  from  a  trauma 
unrelated  to  active  duty,  such  as  a  childhood  trauma. 

A  repeated  theme  in  several  interviews  was:  veterans  not  willing  to  open  up  about  their  mental  health  problems 
during  visits.  Cne  PCP  said: 
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•  “I  was  getting  the  veteran  to  acknowledge  that  they  actually  have  PTSD  or  symptoms  of  PTSD.  To  me,  I 
think,  it’s  the  challenge  of  them,  they’re  in  denial.  They’re  in  a  stage,  they’re  not  ready  to  accept  that  this 
could  possibly  affect  them  and  they  could  possibly  have  the  diagnosis.”  (P1) 

PCPs  explained  how  veterans  are  concerned  over  having  a  mental  health  diagnosis  in  their  record  because  of 
the  stigma  and  impact  on  their  status  in  the  military. 

Other  challenges  included  lack  of  time  to  do  a  mental  health-related  assessment  during  a  medical  visit  and  the 
provider  failing  to  recognize  that  their  patient  is  a  combat  veteran. 

Difficulties  in  treatment  and  management  of  PTSD 

The  respondents  talked  about  the  diversity  of  veterans  with  PTSD  and  other  co-morbidities,  and  acknowledged 
that  it  was  challenging  to  determine  if  some  physical  symptoms  (like  itching,  etc.)  represented  a  physical 
manifestation  of  PTSD.  One  POP  provided  an  example  of  the  challenges  involved  in  treating  and  managing 
PTSD  in  homeless  people  with  PTSD  who  are  also  substance  users. 

One  POP  commented  on  how  it  can  be  challenging  to  help  veterans  see  the  connection  between  some 
symptoms  like  insomnia  due  to  nightmares  and  PTSD.  Another  POP  elaborated  on  the  issue  of  veterans 
refusing  treatment  for  PTSD  due  to  the  stigma  of  help-seeking  and  mental  health  treatment  in  general.  One  POP 
felt  strongly  that  medical  and  psychological  management  of  PTSD  can  bring  relief,  but  is  not  sufficient  and 
suggested  that  many  veterans  need  a  more  comprehensive  solution  to  addressing  their  overall  health  problems 
and  well-being. 

Factors  influencing  PTSD  management 

Nearly  half  of  respondents  (4  of  10)  talked  about  PCPs  having  insufficient  training,  experience,  or  exposure  with 
respect  to  mental  health  problems.  One  POP  noted  that  VA  does  not  pay  for  Mindfulness  Based  Stress 
Reduction  (MBSR)  training  for  veterans. 

Other  reported  factors  that  can  negatively  impact  PTSD  management  included: 

•  Financial  and  policy  factors  that  limit  treatment  and  prescribing  options. 

•  Socioeconomic  factors,  including  lack  of  stable  housing. 

•  Not  having  access  to  useful  screening  tools  and  other  tools  to  help  detect  mental  health  problems 

•  Inconsistent  staffing  and  changes  in  services  may  result  in  veterans  receiving  inaccurate  information 
about  treatment  options  available  to  them 

•  Many  providers  do  not  have  a  way  to  de-compress  after  dealing  with  difficult  and  emotionally  draining 
patients;  this  is  not  a  part  of  the  culture  of  primary  care. 

PTSD  Treatments  that  PCP  Participants  Recommend 

When  asked  about  treatments  they  recommend  for  patients  with  PTSD,  PCPs  mentioned  multiple  therapies 
(see  Table  below).  Notably,  none  of  the  interviewed  PCPs  reported  recommending  a  community 
engagement/psychosocial  approach. 


Treatments  Named  by  PCPs 

PI 

P2 

P3 

P4 

P5 

P6 

P7 

P8 

P9 

P10 

Medications 

SSRIs  (anti-depressant  drugs) 

X 

X 

X 

X 

X 

Medication  (Prazosin,  Flouxetine,  Sertraline,  Paxil) 

X 

X 

X 

X 

X 

X 

Steroid  Injections 

X 

Pain  Clinic 

X 

Behavioral  Health 

Cognitive  behavioral  therapy  (CBT) 

X 

X 

Rapid  Processing  Eye  Movement  (EMDR) 

X 

MH  Referrals 

X 

X 

X 

X 

X 

X 

Cognitive  processing  therapy  (CPT) 

X 

Evidence  based  psychotherapy,  specifically 
prolonged  exposure  therapy  (PE) 

X 

X 

Substance  Abuse  Treatment  (AA  or  NA) 

X 
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Treatments  Named  by  PCPs 

P1 

P2 

P3 

P4 

P5 

P6 

P7 

P8 

P9 

P10 

Community  Engagement/Psychosocial  Approach 

Community  Engagement/Psychosocial  Approach 

Mind-Body/Integrative  Treatments 

Mindfulness  Meditation 

X 

X 

Mantram  Repetition,  an  evidence-based  meditation 
technique  tested  at  VA  with  HSR&D  funding* 

X 

Relaxation  and  joyfulness  techniques** 

X 

Acupuncture 

X 

X 

X 

Chiropractic  Care 

X 

Podiatry 

X 

Yoga 

X 

Vet  Center  Services  (Alpha-Stim) 

X 

Environmental  Factors*** 

Safe,  decent,  and  beautifui  housing 

X 

*  Mantram  Repetition  is  an  evidence-based  meditation  technique  adapted  to  VA  by  Jiii  Bormann  at  San  Diego 
VA.  The  book  “Strength  in  the  Storm:  Transform  Stress,  Live  in  Baiance,  and  Find  Peace  of  Mind,"  outiines  this 
practice  in  detaii. 

**Possibiy,  respondent  referred  to  “Acceptance  and  Commitment  therapy” 

***  Environmentai  factors  are  not  necessariiy  covered  in  the  curricuium 

Practice  Patterns  and  Use  of  Motivational  Interviewing  (Ml)  and  Shared  Decision-Making 

Severai  PCPs  eiaborated  on  their  practice  patterns  inciuding  communication  with  patients,  a  generai  approach 
to  care  for  patients  with  PTSD,  and  referrais.  Aii  but  one  PCP  reported  using  motivationai  interviewing  (Ml)  in 
their  practice.  One  PCP  reported  having  in-depth  discussions  of  treatment  options  with  patients.  The  same  PCP 
reflected  on  a  hoiistic  approach  to  care  for  patients  with  PTSD.  Another  PCP  reported  routineiy  doing  the  suicide 
risk  assessments  and  often  referring  to  a  mentai  heaith  provider,  and  one  PCP  reported  a  preference  to  ieave 
medication  prescribing  to  a  speciaiist  in  certain  situations,  such  as  in  patients  having  nightmares. 

Severai  interview  questions  addressed  Ml.  In  addition  to  use  of  Ml  in  practice,  respondents  reflected  on  their  Ml 
training  experience.  Two  had  noformai  training  in  Ml,  five  participated  in  Ml  course(s)  provided  by  various 
sources,  and  three  were  experts  in  Ml  as  they  reported  being  either  a  faciiitator  in  Ml  training  or  an  investigator 
in  an  Ml  training  study. 

The  iist  beiow  summarizes  Ml  techniques  that  were  identified  by  the  respondents  as  usefui  in  their  practice 
(using  the  terms  stated  by  the  interviewed  PCPs): 

•  Expioring  patient’s  beiiefs,  asking  them  to  expiain  the  risks/benefits  of  a  particuiar  behavior, 
juxtaposing  patient  behavior  with  ideai  heaith  (three  PCPs) 

•  Reflecting  veterans’  words,  vaiues,  and  preferences  back  to  them  for  discussion  (three  PCPs) 

•  Open  ended  questions  (two  PCPs) 

•  Affirmations  (two  PCPs) 

•  Readiness/confidence  ruier  (two  PCPs) 

•  Repeating  back  to  veterans  what  she/he  is  hearing  (one  PCP) 

•  Heiping  veterans  set  their  own  goais  (one  PCP) 

•  Expressing  empathy  (one  PCP) 

One  PCP  acknowiedged  the  chaiienge  of  knowing  what  Ml  technique  to  use  at  what  time.  Other  PCPs  reported 
choosing  more  simpie-to-use  Ml  techniques  and  deciding  whether  to  use  Ml  at  all  or  not  based  on  the 
relationship  with  the  veteran.  One  PCP  saw  the  benefit  of  involving  other  members  of  a  care  team  in  using  Ml 
with  veterans,  but  it  was  not  part  of  their  practice  at  the  time  of  the  interview. 

A  question  about  provider-patient  shared  decision-making  was  asked  in  four  interviews.  Shared  decision-making 
was  explained  differently  by  the  PCPs,  including  one  appropriate  definition  of  shared  decision-making  and 
another  definition  that  likely  reflected  a  misconception  about  this  approach.  One  PCP  noted  that  shared 
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decision-making  is  time-consuming,  and  one  PCP  reported  iack  of  skiiis  and  confidence  necessary  for  use  of 
shared  decision-making. 

Suggestions  Focused  on  the  VW  Training  Content 

Three  PCPs  made  suggestions  reiated  to  the  proposed  training  content  at  various  points  in  the  interview.  These 
suggestions  are  summarized  beiow: 

•  Inciude  different  kinds  of  trauma — chiidhood  trauma,  sexuai  trauma,  other  abuse  in  addition  to 
combat  trauma 

•  Make  the  training  easy  to  understand  and  not  compiicated 

•  Inciude  information  on  how  fear/anxiety  is  a  driving  force  for  chronic  iiiness;  prior  trauma  might 
iower  the  threshoid  for  experiencing  fear/anxiety  and  make  these  feeiings  more  common. 

•  Inciude  psychosociai  aspects  of  iiiness 

•  Address  the  concept  of  forgiveness  (i.e.,  heiping  the  soidiers  to  recognize  that  what  they  are  going 
through  is  not  their  fauit). 

•  Inciude  how  to  express  gratitude  to  veterans  for  their  service  to  the  country. 

•  Make  sure  that  providers  understand  that  PTSD  treatment  must  be  taiiored  to  the  individuai 
depending  on  their  symptoms,  goais  and  vaiues  and  particuiar  iife  circumstance. 

•  Address  provider  secondary  “trauma”  reiated  to  treating  patients  with  PTSD 


Training  Deiivery:  Feasibiiity  and  Acceptabiiity 
Acceptance  of  Virtuai  Worid  Training 

Haif  of  PCPs  reported  high  acceptance  of  virtuai  reaiity  as  the  proposed  training  modaiity,  citing  that  it  has  the 
potentiai  to  heip  providers  iearn  and  may  repiace  some  of  the  current  training  modaiities  in  the  future.  One  PCP 
said: 

•  “When  I  watched  the  video  ciip,  I  was  pretty  amazed  because  it  did  pretty  much  iook  iike  a  video 
game  and  the  idea  of  having  simuiated  patients  or  even  just  a  ciassroom  in  the  educationai 
environment,  it  seems  very  ciever  and  couid  be  additive  or  even  take  the  piace  of  some  of  the 
training  that  we  get  in  the  medicai  fieid.”  (P7) 

Four  interviewees  beiieved  that  the  VW  training  wiii  be  weii-received  by  younger  PCPs.  For  exampie: 

•  “I  think.  I’m  going  to  sound  oid,  but  I  think  this  is  a  necessary  thing  for  the  miiienniais  and  the  peopie 
who  are  training  now.  I  think  for  someone  more  in  the  baby  boomer-ish  era,  it  wouid  be  a  iittie  bit 
more  taxing.  I  found  when  I  iooked  at  the  video  I  thought  This  wouid  be  reaiiy  cooi  for  my  kids,  but 
this  might  make  me  crazy.’  Nobody  does  anything  without  a  phone  these  days,  inciuding  myseif, 
you  know  peopie  are  used  to  growing  up  gaming  and  doing  those  things  and  I  think  that  peopie  who 
are  very  comfortabie  in  that  environment,  it  wouid  be  a  fabuious  training.”  (P4) 

•  “I  think  it’s  a  generationai  thing.  I  think  for  my  generation.  I’m  almost  55.  We  didn’t  grow  up  with  this. 
So  it  has  been  a...,  at  first  I  was  cynical  and  now  I  am  much  more  open.”  (P9) 

Several  PCPs  noted  that  a  VW  training  would  be  convenient,  because  it  is  online  interesting  because  of  its 
novelty,  and  possibly,  cost  effective.  Interactivity  among  VW  users,  opportunities  to  provide  immediate 
feedback,  and  capability  to  support  simulated  patient  experience  were  viewed  as  positive  features  of  VW.  One 
PCP  felt  that  VW  may  be  used  to  build  a  sense  of  community  among  providers.  Another  PCP  elaborated  on  the 
possibility  of  converting  a  face-to-face  training  into  the  VW  training,  and  one  noted  that  such  training  would  be 
good  for  providers  who  are  new  to  the  VA  system. 

Four  PCPs  were  skeptical  regarding  the  VW  training  as  they  did  not  see  the  value  of  this  training  modality  or 
considered  the  VW  training  to  be  overwhelming.  For  example,  one  PCP  questioned  the  value  of  the  VW  training: 

•  “I  guess  I’m  wondering  what  is  the  added  value  over  like  some  similar  like  structures  like  role  playing  or 
even,  I  think  things  like  taping  your  encounters  or  standardized  patients  or  something  like  that.  It  seems 
like  a  lot  of  effort  to  create  something  like  that  and  what  is  the  additional  value  of  that  over  some  of 
these  other  things...  I’m  a  little  skeptical...!  mean  I  love  the  idea  of  supporting  primary  care  doctors  in 
learning  more  about  PTSD  and  getting  more  comfortable  with  it,  so  if  that’s  a  way  to  make  that  work.” 
(PS) 
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Two  PCPs  felt  that  audience  generation  for  the  VW  training  would  be  a  challenge. 

Notably,  interviewed  PCPs  had  no  or  very  limited  exposure  to  VW,  and  some  of  their  statements  revealed 
misconceptions  about  VW.  For  example,  one  PCP  thought  that  VW  participants  would  need  virtual  reality 
goggles  to  participate  in  the  training. 

Suggestions  Focused  on  the  VW  Training  Deiivery 

Six  PCPs  made  suggestions  related  to  the  mode  of  training  delivery  which  are  listed  below: 

•  Make  training  fun  and  engaging 

•  Avoid  making  this  training  formulaic  as  some  VA  trainings  have  been  in  the  past 

•  Make  the  training  similar  to  previous  trainings  participants  have  done 

•  Make  the  training  user-friendly/avoid  technical  glitches  in  virtual  environment 

•  Create  modules  in  18-20  minute  blocks 

•  Make  the  training  short 

•  Provide  immediate  feedback  to  providers  learning  new  Ml  skills 

•  Use  case-based  learning 

•  Use  a  TED  Talk  format 

•  Find  research  to  support  high  information  retention  rates  through  this  training  modality 

•  Make  the  training  efficient  for  providers 

•  Highlight  the  importance  of  the  training 

•  Be  selective  with  the  initial  participants  to  increase  chance  of  getting  positive  feedback 

Barriers  and  Facilitators  for  Participation  in  VW  Training 
Facilitators 

Several  PCPs  thought  that  having  dedicated  time,  in  particular,  having  blocked  clinic  time  would  support 
participation  in  the  VW  training.  Half  of  PCPs  commented  on  CME  credit,  indicating  that  it  is  an  incentive,  but 
likely  not  a  big  draw  to  participation.  Evidence  of  positive  educational  impact  was  viewed  as  participation 
facilitator  by  two  PCPs.  Two  other  participants  talked  about  positive  feedback  and  testimonials  from  participants 
as  factors  that  may  increase  future  participation.  Other  facilitators  reported  by  PCPs  included  relevance  to 
practice;  opportunity  to  learn  something  new;  being  able  to  do  training  from  the  VA  or  at  home;  desktop 
computer  and  mobile  access  to  training;  quality  improvement  credit/incentive;  self-paced  training;  user- 
friendly/fun  to  use  technology;  novelty/technology  coolness;  non-judgmental  environment;  training  being  free; 
gift  card/purchasing  VR  goggles  for  participants.  One  PCP  mentioned  food,  not  explaining  how  this  incentive 
may  work  in  VW. 

Barriers 

Four  PCPs  emphasized  the  importance  of  finding  time  for  the  training,  with  one  PCP  also  talking  about  the  time 
to  create  an  account  and  learn  how  to  use  the  program.  Several  PCPs  named  fear  of  technology  and 
computer/access  problems  as  the  barriers  to  participation.  Additional  barriers  named  by  PCPs  included 
resistance  because  the  training  is  new  and  different,  low  satisfaction  with  previous  VA  trainings,  participant  fee, 
and  older  age. 

One  PCP  elaborated  on  the  barriers,  saying: 

•  “But  it  is  one  of,  sort  of  the  very,  ‘work-a-day’  barriers  of  physicians  having  time  in  their  day  to  actually 
having  the  minutes  to  participate,  having  the  bandwidth  to  get  an  e-mail  and  actually  read  it  and  then 
track  the  information  that  is  needed  to  build  the  access  to  the  program.  What  you’re  describing,  virtual 
reality  seems  several  generations  beyond  the  current  level  of  functionality  of  VA  related  IT.  And  maybe 
then,  the  flip  side  of  that  coin  is  that,  it’s  exciting  and  different  in  a  way  that  really  sets  it  apart  from  other 
opportunities  for  training.  I  think  a  real  challenge,  I  imagine  VA,  it’s  been  a  long  time  since  I’ve  read  the 
Scarlett  Letter,  but  that’s  what  comes  to  mind  for  me,  you  know  VA  trainings  wear  a  badge  of  a  painful 
use  of  3  hours  of  your  day  when  you  otherwise  could  be  doing  other  things.  I  do  think  a  challenge  to  this 
will  be  identifying  it  as  a  training  related  to  VA.  I  think  it  travels  in  rough  company  in  that  regard.” 
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Generalizability/Applicability  to  Broader  Audience 

When  asked  about  applicability  of  the  described  VW  training  to  the  broader  PCP  audience,  one  PCP  stated  that 
such  training  might  not  be  generalizable  because  the  VA  experience  is  different  from  the  experience  in  the  non- 
VA  settings.  Answering  the  same  question,  other  PCPs  offered  suggestions  for  how  to  make  the  VW  training 
applicable  to  the  broader  audience.  Some  suggestions  overlapped  with  those  for  the  training  content  and 
delivery  stated  earlier  in  the  report.  The  suggestions  included: 

•  Include  childhood  trauma,  sexual  trauma,  abuse,  and  other  types  of  non-combat  trauma 

•  Reflect  the  provider’s  most  common  type  of  patients 

•  Reflect  PTSD  with  comorbid  mental  health  issues 

•  Include  range  of  patients  in  age,  experience,  and  treatment  options 

•  Have  profession-specific  training  modules 

•  Keep  the  material  down  to  a  bare  irreducible  minimum  (BIM)  when  creating  content,  then,  build  off  the 
BIM 

•  Keep  the  training  relevant  with  current  events  in  the  medical  world;  be  sensitive  to  the  broad  range  of 
care  issues  that  PCPs  have  to  deal  with  each  day 

•  Use  a  stronger  form  of  training  than  passive  learning 

•  Vary  participant’s  age  ranges  for  the  initial  testing  group  to  create  ambassadors  for  the  program 

•  Show  educational  impact 


Discussion 

Training  on  PTSD  for  PCPs  is  Reievant  and  Needed 

The  interviewed  PCPs  acknowledged  multiple  challenges  and  factors  influencing  diagnosis,  treatment  and 
management  of  PTSD.  Many  of  these  are  consistent  with  findings  from  prior  research.  PTSD  manifests  itself  in 
different  ways  and  frequently  co-occurs  with  other  mental  health  conditions,  including  a  broad  range  of 
substance  use,  mood,  anxiety,  and  personality  disorders  (Back  et  al,  2014;  Goldstein  et  al,  2016).  Half  of 
participants  stated  that  this  presents  a  challenge  because  clinical  manifestations  of  comorbid  conditions  may  be 
similar  to  PTSD  or  found  to  be  symptoms  of  PTSD.  As  noted  by  one  participant,  differentiating  PTSD  from  other 
mental  health  conditions  is  especially  difficult  in  older  patients,  and  it  is  known  that  more  than  60%  of  military 
veterans  in  the  United  States  are  55  years  or  older  and  older  age  is  associated  with  a  higher  likelihood  of 
reactivated  or  delayed-onset  PTSD  (Mota  et  al,  2016).  Participants  also  noted  that  it  is  difficult  to  recognize 
actual  trauma  from  imagined  trauma  due  to  dementia,  psychosis,  and  other  issues,  and  recognize  the  origin  of 
the  trauma,  which  may  be  trauma  related  to  active  duty  or  trauma  caused  from  other  experiences,  such  as 
abuse  in  childhood,  adult  emotional  or  sexual  trauma. 

Participants  talked  about  limitations  of  screening  tools  as  a  challenge,  such  as  false-negative  results.  It  is 
possible  that  improved  screening  instruments  will  address  some  of  the  limitations,  as  current  tools,  the  Primary 
Care  PTSD  screen  (PC-PTSD)  and  the  PTSD  Checklist,  are  being  examined  and  modifications  are  suggested 
to  reflect  the  new  Diagnostic  and  Statistical  Manual  of  Mental  Disorders  (DSM-5)  criteria  for  PTSD  (Prins  et  al, 
2016;  Spoont  et  al,  2015).  At  the  same  time,  implementation  of  updated  screens  in  routine  practice  would  create 
additional  need  for  education. 

Another  challenge  reported  during  interviews  was  lack  of  time  during  a  medical  visit  to  do  a  mental  health- 
related  assessment  of  a  veteran  patient.  This  challenge  is  discussed  in  the  literature.  For  example,  lack  of  time 
to  deal  with  psychologic  problems  was  named  as  a  major  barrier  to  PTSD  care  by  66%  of  PCPs  in  a  study  done 
by  Meredith  and  colleagues  (Meredith  et  al,  2009) 

According  to  participants,  veterans  are  sometimes  unwilling  to  open  up  about  their  mental  health  problems 
during  the  visit  with  providers  and  this  can  increase  the  difficulty  in  recognizing  PTSD,  providing  treatment,  or 
referring  to  treatment.  Participants  explained  that  one  reason  for  this  unwillingness  is  a  stigma  associated  with  a 
mental  health  diagnosis.  Veterans  express  concern  over  the  impact  of  having  a  PTSD  diagnosis  in  their  file. 
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This  is  sometimes  tied  to  concerns  over  how  such  a  diagnosis  wiii  affect  their  status  in  the  miiitary.  Stecker  and 
coiieagues  (Stecker  et  ai,  2013)  studied  beiiefs  of  veterans  who  screened  positive  for  PTSD  and  found  that 
decision  not  to  seek  treatment  for  symptoms  of  combat-reiated  PTSD  was  influenced  by  stigma  in  16%  of  cases. 
Beiiefs  about  stigma  feii  into  two  categories — inciuding  seif-stigma  and  the  idea  that  treatment  wouid  resuit  in 
consequences.  This  study  identified  other  beiiefs  that  created  barriers  to  treatment,  which  did  not  emerge  in  our 
formative  evaiuation.  Those  inciuded  concerns  about  treatment  (40%),  emotionai  readiness  for  treatment  (35%), 
and  iogisticai  issues  (8%). 

In  our  formative  evaiuation,  neariy  haif  of  participants  thought  that  that  PCPs  had  insufficient  training, 
experience,  or  exposure  with  respect  to  mentai  heaith  probiems.  This  is  consistent  with  the  findings  of  a  survey 
study  of  PCPs  in  community  heaith  centers,  where  PCPs  indicated  insufficient  knowiedge  about  PTSD 
diagnosis  (28%  of  respondents)  and  insufficient  knowiedge  about  PTSD  treatment  (27%  of  respondents)  as  the 
major  barriers  to  PTSD  care  (Meredith  et  ai,  2009).  The  chaiienges  associated  with  PTSD  care  and  PCPs’  iack 
of  training  and  knowiedge  in  this  ciinicai  area  support  the  reievance  and  need  for  the  training  that  is  being 
deveioped  through  this  project. 


Treatments,  Practice  Patterns  and  Use  of  Motivational  Interviewing 

When  asked  about  which  types  of  treatment  they  recommend,  PCPs  overwheimingiy  named  pharmaceuticai 
interventions,  such  as  anti-depressants  and  prazosin,  a  drug  for  patients  with  nightmares.  Interestingiy,  the 
second  most  popuiar  treatment  recommendation  for  veterans  presenting  with  PTSD  symptoms  was  referrai  to 
mentai  heaith.  Whiie  these  two  approaches  are  supported  by  evidence,  they  both  focus  on  psychoiogicai 
treatments,  rather  than  recommend  more  integrative  treatments,  targeting  both  mind  and  body. 

One  surprising  finding  was  that  not  one  of  the  10  PCPs  mentioned  community-based  approaches  to  treatment 
for  PTSD.  Deveioping  research  has  demonstrated  that  traditionai  treatments  for  PTSD  have  oniy  iimited 
effectiveness.  Modaiities  once  thought  to  be  the  goid-standard  of  treatment,  such  as  proionged  exposure 
therapy,  are  no  ionger  viewed  as  such  (Neyian,  et.  ai.,  in  pubiication).  Aiternativeiy,  a  panei  of  highiy 
specialized  physicians  recommended  engagement  in  social  and  vocational  activities  as  a  means  of  not  just 
temporarily  alleviating  PTSD  symptoms,  but  as  a  long-lasting  method  of  coping. 

Current  trends  in  the  VA  health  care  system  include  a  shift  to  a  Whole  Health  Approach  (VA  Patient  Centered 
Care  website)  to  care  for  veteran  patients.  This  involves  a  personalized,  proactive,  patient-driven  model  of 
healthcare,  which  is  supported  by  positive  relationships  between  provider  and  patient.  Motivational  Interviewing 
can  enhance  this  relationship  and  allow  PCPs  to  better  understand  their  veteran  patients’  values  and  health 
goals.  Research  has  shown  that  patients  are  more  likely  to  demonstrate  readiness  for  change  and  are  less  likely 
to  drop  out  of  interventions  in  studies  where  Ml  was  used  (Blain  2013). 

However,  even  the  most  seasoned  professionals  may  not  be  consistent  in  utilizing  Ml  in  their  practice.  The 
interviews  revealed  a  diverse  list  of  Ml  techniques  that  were  implemented,  but  also  addressed  some  challenges, 
such  as  knowing  when  to  use  Ml  with  a  patient.  Regardless  of  prior  training,  PCPs  expressed  varying  levels  of 
comfort  with  either  implementing  or  recommending  certain  types  of  care.  This  demonstrates  a  need  for  the 
curriculum  to,  therefore,  focus  on  building  the  self-efficacy  of  PCPs  to  present  options  to  their  patients,  rather 
than  encouraging  PCPs  to  implement  these  treatments  in  their  practice. 

Opinions  About  Acceptance  of  VW  Training  Varied 

Opinions  of  interviewed  PCPs  about  VW  training  varied  from  seeing  this  format  as  highly  desired  for  peer  PCPs 
to  conservative  and  even  negative  opinions  about  the  acceptance  and  value  of  education  in  VW.  These  opinions 
should  be  interpreted  with  caution  because  participants  lacked  VW  experience  and  understanding  of  VW 
capabilities.  At  the  same  time,  participants  seemed  to  be  in  agreement  recognizing  the  value  of  interactive 
educational  approaches  and  feedback,  which  are  recognized  as  effective  continuing  education  strategies 
(Moore  et  ai,  2009),  and  some  acknowledged  that  these  strategies  can  be  used  in  VW.  Based  on  discussions 
with  the  VW  experts  who  are  the  part  of  the  project  team,  opinions  of  clinicians  who  are  not  familiar  with  VW  are 
likely  to  change  once  they  have  exposure  to  the  VW  training;  they  may  find  VW  more  in-depth  and  immersive 
than  they  originally  thought  or  anticipated. 
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Several  participants  speculated  that  younger  PCPs  would  better  accept  a  VW  training  than  their  older  peers.  It  is 
a  common  belief  that  3D  virtual  environments  are  particularly  attractive  for  the  younger  generation  who 
frequently  and  naturally  use  digital  technologies  in  everyday  life  (Hunsinger  et  al,  2012).  However,  multiple 
studies  conducted  in  different  countries  suggest  that  the  “digital  native”  label  does  not  provide  evidence  of  a 
better  use  of  technology  to  support  learning,  because  other  factors  related  to  learner  characteristics  and 
teaching  model  are  also  important  or  more  important  in  this  respect  (Gros  et  al,  2012). 

An  important  topic  of  interest  to  the  research  team  was  how  to  increase  the  participant’s  willingness  to  engage 
in  the  training.  When  asked  about  this,  participants  noted  that  the  training  needed  to  be  advertised  as  fun, 
exciting,  and  different  from  a  traditional  VA  training.  More  specifically,  participants  stated  that  the  training  should 
not  be  formulaic  as  some  VA  trainings  have  been  in  the  past.  Collectively,  participants  named  many  factors  that 
were  either  barriers  or  facilitators  for  PCPs  participation  in  a  VW  training.  While  some  of  the  factors  are  outside 
of  the  education  planners’  control,  such  as  participants  having  dedicated  time  for  training,  other  factors  should 
be  addressed  by  the  planners,  including  access  to  the  training  from  VA  computers,  learner  support,  and 
mitigating  and  resolving  technical  issues,  which  are  to  be  anticipated  (Shershneva  et  al,  2014). 

Implications 

Below  is  a  summary  of  implications  for  the  VW  training  development  and  implementation  that  the  research  team 
drew  from  the  analysis  of  the  PCP  interview  data.  Several  implications  supported  the  training  features  that  have 
been  planned  or  considered,  such  as  emphasizing  the  role  of  PCPs  in  PTSD  care  and  resolving  issues  related 
to  access  to  training  within  the  VA  facilities.  Some  implications  are  not  specific  to  the  capabilities  of  VW  and 
reflect  the  best  practices  in  continuing  education,  such  as  use  of  case-based  learning,  role-playing,  and 
feedback.  The  evolving  curriculum  already  includes  these  elements.  By  contrast  with  these  elements,  some 
opinions  and  suggestions  from  the  interviewed  PCPs  were  critically  reviewed  and  not  reflected  in  the  list  below 
because  they  were  viewed  less  relevant  to  the  current  training-in-development,  such  a  suggestion  to  use  TED- 
style  presentations.  Notably,  the  emerging  results  and  implications  were  documented  in  the  matrix  table,  and 
evolving  versions  of  the  matrix  table  were  shared  and  reviewed  with  the  PI  and  the  curriculum  team  to  discuss 
their  applicability  to  the  VW  training-in-development. 

Training  deveiopment  and  evaiuation: 

•  Add  evaluation  questions  about  perceived  complexity  of  training 

•  Have  a  sample  of  learners  that  is  diverse  in  age,  training,  location,  etc. 

o  Particularly  important:  should  pay  attention  to  age  during  randomization  process  for  focus  groups. 

•  Review  interview  notes  when  recruiting  training  participants  to  identify  candidates  (e.g.,  P4) 

Audience  generation: 

•  Given  that  some  clinicians  believe  they  are  using  Ml  in  their  practice,  the  audience  generation  strategies 
may  need  to  have  the  language  about  advancing  Ml  skills  rather  than  introduction  to  Ml 

•  Advertising  should  convey  precisely  what  the  learner  can  expect  from  the  training 

•  Clearly  state  technical  requirements  for  participation 

•  Offer  CME  credit,  as  it  is  a  desired  feature  of  the  training 

•  Collect  (positive)  testimonials  from  VW  training  participants  to  use  for  generation  of  future  audience 

•  Consider  including  evidence  of  the  effectiveness  of  education  in  VW.Training  in  the  training 
description/audience  generation  materials 

•  Consider  presenting  training  to  potential  participants  as  fun,  engaging  and  valuable 

Content: 

•  Reflect  in  the  curriculum  that  different  kinds  of  trauma  may  lead  to  PTS  in  veterans 

•  Address  community  engagement/psychosocial  approaches 

•  Present  co-morbid  mental  health  issues 

•  Include  psycho-social  aspects  of  illness 

•  Discuss  disrupted  fear  networks 

•  Invest  sufficient  time  in  explaining  clinical  manifestations  of  PTSD 

•  Provide  learners  with  downloadable  practice-oriented  tools  and/or  links  to  tools  (e.g.,  tools  existing  within  VA 
HER/system) 
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•  Demonstrate  to  learns  how  best  practices  in  screening  for/diagnosis  of  PTS  and  managing  of  patients  with 
PTSD  may  be  time  efficient 

•  Emphasize  the  role  of  PCPs 

•  Reframe  the  post-traumatic  stress  symptoms  to  not  imply  a  “disorder” 

•  Consider  providing  a  definition  of  shared  decision-making 

•  For  learners  from  Kaiser  and  community  health  care  settings:  consider  providing  tips  for  interacting  with 
veteran  patients. 

Delivery: 

•  Revisit  IT  logistics  to  ensure  VA  access  to  the  training 

•  Create  a  safe,  non-judgmental  environment  to  practice  skills 

•  Provide  synchronous  training 

•  Use  case-based  learning  and  role-playing 

•  Support  building  community  among  PCPs,  provide  opportunities  for  learner-learner  interaction  and 
networking 

•  Provide  immediate  feedback  on  learner  performance 

•  Advocate  for  potential  learners  to  have  designated  days/times  for  education  in  their  setting  and  consider 
these  designated  times  when  scheduling  VW  training  sessions 

•  Provide  learner  support  to  address  computer  problems 

•  Involve  VA  training  experts  for  insight 

Generalizability 

•  Consider  the  implications  for  content  and  delivery  stated  above  as  means  to  increase  applicability  of  the  VW 
training  to  the  broader  audience  of  PCPs 

•  Consider  tailoring  the  training  content  to  the  educational  needs  and  experience  of  PCPs  practicing  in  the 
non-VA  settings 

•  Emphasize  interprofessional  collaborative  practice  as  related  to  care  for  patients  with  PTSD 

Limitations 

We  used  a  convenience  sample  of  seasoned  PCPs  affiliated  with  VA  and  results  cannot  be  generalizable  to  the 
broader  population  of  PCPs  affiliated  with  the  VA  system  and  the  population  of  PCPs  practicing  outside  of  the 
VA  system.  However,  generating  generalizable  results  was  not  the  purpose  of  this  study  because  it  was 
formative  evaluation  to  inform  training  development  and  not  a  research  study. 

Lack  of  participant  familiarity  with  VW  environment  and  a  choice  some  participants  made  to  not  view  the 
provided  video  about  education  in  VW  prior  to  the  interview  led  to  participant  responses  based  on  insufficient  or 
inaccurate  understanding  of  the  VW  capabilities. 


Conclusion 

1 .  Findings  about  challenges  in  primary  care  practice  related  to  providing  care  to  veterans  with  PTSD  as  well 
as  treatment  choices  by  PCPs  and  use  of  motivational  interviewing  are  consistent  with  the  published 
studies. 

2.  Findings  support  the  need  for  training  for  PCPs  focused  on  PTSD  and  Ml  applied  to  providing  care  for 
patients  with  PTSD. 

3.  Opinions  of  interviewed  PCPs  about  VW  training  were  limited  to  their  insufficient  understanding  of  VW. 

4.  This  formative  evaluation  generated  implications  for  audience  generation,  content,  and  training  delivery  and 
evaluation,  and  training  generalizability  that  are  being  utilized  for  the  VW  training  development  in  this 
project. 
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Appendices 

Appendix  1:  Interview  Guide 


Before  the  Interview 

Participants  wiii  receive  background  information  about  the  Virtuai  Worids  project  prior  to  the  interview,  e.g., 
curricuium  info. 

Introduction 

[Confirm  that  participant  has  received  the  approved  Information  Sheet.] 

First,  I  want  to  thank  you  for  taking  the  time  to  taik  with  me  today!  The  purpose  of  this  brief  interview  is  to  get 
some  input  from  you  based  on  your  ciinicai  experience  as  a  primary  care  provider.  Our  team  is  deveioping  a 
new  training  intervention  for  primary  care  providers  to  improve  their  abiiity  to  identify  and  manage  posttraumatic 
stress.  Whiie  there  are  other  interventions  for  posttraumatic  stress,  the  one  we  are  pianning  is  interactive  and  it 
wiii  be  heid  in  an  oniine  environment.  So,  the  overaii  purpose  of  our  conversation  today  wiii  be  to  heip  me 
understand  some  of  your  needs  as  a  ciinician  around  posttraumatic  stress  and  your  thoughts  about  the  oniine 
environment. 

Your  participation  in  this  interview  is  voiuntary.  You  may  choose  to  not  answer  any  question  or  stop  the 
interview  at  any  time.  Do  you  have  questions  for  me  before  we  begin? 

[Recite  Audio  Consent  ianguage  from  the  protocoi.  START  AUDIO  RECORDER.] 

Domain  1:  Needs  Assessment  of  Proposed  Training  Topics  —  Focus  on  Curriculum 

Q1 .  In  your  everyday  clinical  practice,  what  are  some  of  the  challenges  recognizing  posttraumatic  stress 

(PTS)? 

Q2.  What  are  some  typical  treatments  you  have  recommended  for  veterans  with  PTS? 

Follow-up:  Are  there  other  treatments  you  have  heard  about,  but  don’t  usually  recommend?  If  not,  why 
not? 

Q3:  The  VA  requires  PCPs  to  have  basic  training  in  motivational  interviewing.  Can  you  tell  me  about 
your  experience  using  motivational  interviewing? 

Follow-up:  What  have  been  some  of  the  challenges  you  have  encountered  to  using  motivational 
interviewing  skills  in  your  everyday  practice? 

Q4.  What  role  has  shared  decision-making  played  when  selecting  treatment  for  PTS? 

Domain  2:  Feasibility  and  Acceptability  (Focus  on  Training  Delivery  in  Virtual  World) 

One  of  the  things  I’m  interested  in  is  how  interested  you  might  be  in  participating  in  a  training  that  is  held  in  an 
online  environment.  Did  you  have  a  chance  to  look  at  the  materials  I  sent  before  our  interview  (e.g.,  the  Virtual 
World  introductory  module)? 

IF  YES:  The  idea  behind  our  training  is  that  participants  would  log  into  a  virtual  environment  and  conduct  the 
training  interactively  inside  that  world. 

IF  NO:  It’s  no  problem  if  you  didn’t.  Are  you  at  a  computer  right  now?  If  it’s  OK,  I’ll  send  an  e-mail  with  a  link  that 
I’d  like  for  you  to  click,  so  I  can  get  your  reaction.  [SEND  LINK] 

IF  NO,  Explain: 

A  virtual  world  is  a  computer-based,  3-D,  360-degree  simulated  environment.  It  resembles  a  first-person 
computer  game,  in  which  the  user’s  digital  self  (or  representation)  is  free  to  move  around  an 
environment  at  will,  interacting  with  other  people’s  digital  selves  and  with  the  objects  placed  within  the 
environment.  A  digital  self  may  resemble  the  user’s  appearance  or  may  look  completely  different.  Users’ 
digital  selves  make  gestures,  move,  sit,  and  interact  with  the  environment  in  real  time.  Users  can 
communicate  with  one  another  either  by  speaking  through  a  microphone,  or  by  typing  inside  a  chat 
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system  sort  of  like  an  instant  message  or  text.  When  a  user  talks,  her  or  his  digital  self  talks,  too,  and 
can  be  heard  by  other  users’  digital  selves  positioned  nearby.  A  virtual  environment,  which  can  be 
anything  from  a  beach  to  a  library,  is  usually  expressed  through  color-  and  detail-rich  graphics. 

Q6.  Do  you  have  experience  navigating  in  a  virtuai  worid  for  business,  education,  or  personai  reasons? 

Q7:  From  what  we  have  discussed,  what  do  you  think  about  hoiding  a  medicai  education  training  in  a 
virtuai  environment? 

Domain  3:  Barriers  and  Faciiitators 

As  I  mentioned,  the  long  term  goal  of  this  project  is  to  better  equip  providers  to  work  with  and  treat  patients  who 
have  posttraumatic  stress. 

Q8.  Do  you  think  other  primary  care  providers  wouid  be  wiiiing  to  participate  in  a  training  heid  in  a 
virtuai  environment? 

Follow-up:  Can  you  anticipate  possible  problems? 

Follow-up:  Do  you  have  suggestions  about  how  we  could  make  the  training  user  friendly? 

Follow-up:  Can  you  think  of  something  that  will  make  the  training  relevant  to  busy  primary  care 
providers? 

Q9.  Can  you  think  of  some  reasons  that  may  prevent  a  provider  from  participating  in  a  virtuai  worid 
training  environment? 

Q1 0.  Can  you  think  of  some  reasons  that  may  encourage  a  provider  to  participate? 

Q11.  What  about  yourseif,  what  might  infiuence  your  decision  to  participate  in  a  virtuai  worid  training 
environment? 

Q12.  incentives.  $$,  CME  credit,  Qi  credit,  etc. 

Domain  4:  Generaiizabiiity 

Q12.  Providers  come  from  various  backgrounds,  and  our  goai  is  to  make  sure  the  training  is  usefui  to 
PCP  in  various  practice  contexts.  From  your  experience  of  doing  Continuing  Medicai  Education  (CME), 
what  we  do  to  make  the  training  most  usefui  to  a  broad  audience  of  PCPs? 

interview  Ending 

Q13.  is  there  something  eise  reievant  to  our  conversation  that  i  did  not  ask  about,  but  you  wouid  iike  to 
share? 

Q14.  Do  you  have  questions  for  me? 

Thank  you  for  your  time! 

[STOP  AUDIO  RECORDER] 
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Appendix  2:  Matrix  Table:  Interviews  with  Primary  Care  Providers  (n=10) 


Q# 

Interview  Domain 

Implications 

DOMAIN  1:  RELEVANCE  OF  TRAINING  TOPIC  AND  CURRICULUM 

Q1 

Difficulties  in  recoqnizinq  and  diaqnosinq  PTSD 

1 .  Having  enough  time  to  do  MH-reiated  discovery  during  medicai  visit  (P2,  P5) 

2.  PTSD  manifests  in  different  ways;  differentiai  diagnosis  of  PTS  with  substance 
abuse,  anxiety,  aicohoiism,  and  depression  other  mentai  heaith  conditions  is  a 
chaiienge  (P2,  P3,  P4,  P8,  P9) 

•  Difficuity  recognizing  actuai  trauma  from  imagined  trauma  due  to  dementia, 
psychosis,  etc.  (P3) 

•  Oider  PTSD  patients  (P2) 

3.  Differentiating  etioiogy  of  PTSD  (P1 ) 

•  Miiitary  experience  (combat,  miiitary  sexuai  trauma;  women  or  men  can 
experience  muitipie  forms  of  trauma,  etc.) 

•  Exposure  to  other  kind  of  trauma  experience  (chiidhood,  aduit;  emotionai, 
sexuai,  physicai,  MST,  etc.)  (P1,  P4) 

4.  Faiiure  by  the  provider  to  recognize  that  their  patient  is  combat  veteran  (P4) 

5.  Faiiure  by  the  provider  to  understand  that  veterans  don’t  need  to  be  depioyed  to  a 
warzone  to  deveiop  PTSD  (P4) 

6.  Veterans  not  wiiiing  to  open  up  about  MH  probiems  during  visits  (P2,  P5,  P7) 

•  Veterans  wiii  engage  in  ‘deception’  when  answering  questions  by  answering 
‘no’  to  particuiar  questions  when  the  answer  shouid  be  ‘yes’.  (P5) 

•  Veterans  show  concern  over  having  an  MH  diagnosis  in  their  record 
because  of  stigma  and  impact  on  their  status  in  the  miiitary.  (P1 ,  P5) 

7.  Avaiiabie  screening  tooi  has  iimitations  (P5,  P6,  P10) 

•  Four  question  screeners  for  PTSD  that  are  provided  to  providers  are  not 
enough  to  capture  the  entirety  of  the  diagnosis.  (P5) 

•  PTSD  Screener.  (P6) 

•  Negative  PTSD  screens  may  not  be  correct.  (P10) 

-Reflect  in  the  curricuium  that  different  kinds  of 

trauma  may  iead  to  PTS  in  veterans 

-Differentiai  diagnosis  of  PTS  is  a  chaiienge  for  PCP. 

Is  there  a  room  in  the  curricuium  to  address  it? 

-“Lack  of  time”  is  a  universai  barrier.  It  may  be  helpful 
to  demonstrate  to  the  learns  how  best  practices  in 
screening  for/diagnosis  of  PTS  and  managing  of 
patients  with  PTSD  may  be  time  efficient 
-Providing  learners  with  downloadable  practice- 
oriented  tools  and/or  links  to  tools  will  be  helpful 

-  Issue  for  Kaiser/community  PCPs  in  identifying  if 
the  patient  is  a  Vet 

-Invest  sufficient  time  in  explaining  tell-tale 
symptoms. 

-  Emphasize  the  role  of  the  PCP:  not  to  diagnose 
PTSD,  but  rather  to  manage  symptoms  of  post- 
traumatic  stress  to  the  best  of  ability 

Focus  Group:  We  could  add  evaluation  questions 
about  complexity  of  training. 

Q1 

Difficulties  in  treatment  and  manaqement  of  PTSD 

1 .  Veterans  not  wiiiing  to  accept  the  diagnosis  of  PTSD  and/or  refusing  treatment  for 
PTSD  due  to  stigma  of  heip-seeking,  MH  treatment,  etc.  (P1) 

2.  PTSD  in  homeiess  peopie  who  are  substance  users  (P3) 

3.  Recognize  that  ciinicai  symptoms  may  be  influenced  by/associated  with  PTSD,  such 
as  itching  (P1,  P9) 

4.  Getting  the  veteran  to  understand  that  there  is  a  connection  between  some  issues 
and  PTSD  (insomnia  caused  by  nightmares  caused  by  PTSD).  (P4) 

5.  Diversity  of  veterans  with  PTSD  and  other  co-morbidities  (P3) 

6.  Refusing  treatment  for  PTSD  due  to  stigma  of  heip-seeking,  MH  treatment,  etc.  (P1 ) 

7.  Medicai  and  psychoiogicai  management  is  fine,  but  does  not  get  at  the  root  of  the 
probiem.  Medicai/psychoiogicai  treatment  is  often  a  temporary  band  aid,  what  is 

-  Communication  techniques  are  important  for 
provider-patient  interaction. 

-  Again,  explain  that  PCP’s  role  is  not  to  diagnose. 
Reframe  the  symptoms  to  not  imply  a  “disorder” 
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Q# 

Interview  Domain 

Implications 

needed  for  many  veterans  is  a  comprehensive  soiution  to  soiving  MH  probiems. 

(P3) 

Q1 

Factors  influencing  PTSD  management 

1 .  Financiai  and  poiicy  factors  that  piay  into  treatment  options/prescribing  practices. 

2.  Sociai-economic  factors,  housing  (P3) 

3.  Having  usefui/useabie  toois  to  heip  discover  MH  probiems  (P2) 

•  Ex:  IPC  initiai  visit  tempiate  is  a  tooi  that  heips  to  focus  on  MH  for  returning 
veterans  (P2) 

•  Suicide  Screens/Suicide  Risk  Assessments  (P2) 

4.  Inconsistent  staffing.  (P5) 

5.  POP’s  don’t  have  enough  training,  experience,  or  exposure  to  MH.  (P6,  P7,  P8,  P9) 

•  Providers  coming  from  outside  the  VA  don’t  usuaiiy  have  the  training  to  iook 
for  PTSD  or  its  symptoms.  (P7) 

•  Noformai  training  in  recognizing  PTSD.  (P7,  P8) 

6.  VA  does  not  pay  for  Mindfuiness  Based  Stress  Reduction  (MBSR)  training  for 
veterans  (P3) 

-Toois  existing  within  VA  HER/system  may  be 
utiiized  in  the  training 

-Provide  accessibie  resources  for  mindfuiness 
-Question:  how  wiii  we  design  the  screening  activity, 
if  not  using  PC-PTSD  screener? 

Q2 

PTS  treatment  participants  currently  recommend 

Medications 

1.  SSRIs  (anti-depressant  drugs)  for  patients  with  mood  components.  (P1,  P2,  P5,  P8, 
P10) 

2.  Medication  (Prazosin,  Fiouxetine,  Sertraiine,  Paxii)  (P1,  P2,  P4,  P7,  P9,  P10) 

3.  Steroid  Injections  (P10) 

4.  Pain  Primary  Care  Ciinic  (P10)  (this  one  overiaps:  medication,  mind-body) 

Behaviorai  Heaith 

1.  CBT  (cognitive  behaviorai  therapy)  (P1,  P7) 

2.  ENDR  (Rapid  Processing  Eye  Movement)  (P1) 

3.  MH  Referrais  (P2,  P4,  P5,  P6,  P8,  P9,  P10) 

4.  Warm  handoff  or  a  reguiar  referrai  to  MH 

5.  Substance  Abuse  Treatment  (AA  or  NA)  (P2) 

6.  Cognitive  processing  therapy  (P6) 

7.  Evidence  based  psychotherapy,  specificaiiy  iong  exposure  therapy.  (P6,  P10) 

Communitv  Enqaqement/Psvchosociai  Approach 

Mind-Bodv/Inteqrative  Treatments 

1.  Mindfuiness  Meditation  (P1,  P10) 

2.  Mantram  Repetition  ,  an  evidence-based  meditation  technique  (P1) 

•  Ex:  Strength  in  the  Storm:  Transform  Stress,  Live  in  Baiance,  and  Find 

Peace  of  Mind  (book) 

3.  Reiaxation  and  joyfuiness  techniques  (P2) 

-Primary  care  providers  use  a  variety  of  treatments 
but  are  iikeiy  to  have  varying  ievei  of  comfort  with 
different  treatment  modaiities/agents. 

(Keep  in  mind  that  these  are  seasoned 
professionais.) 

-No  communitv  enqaqement/psvchosociai 
aooroaches  were  mentioned  bv  the  interviewees.  It 
mav  be  underutilized  and  should  be  addressed  in  the 
curriculum. 

UNPUBLISHED  DATA  -  NOT  FOR  DISTRIBUTION 


19 


Q# 

Interview  Domain 

Implications 

4.  Acupuncture  (P2,  P4,  P10) 

5.  Chiropractic  Care  (P10) 

6.  Podiatry  (P 10) 

7.  Yoga(PIO) 

8.  Vet  Center  Services  (Aipha-Stim)  (PI ) 

Environmentai  Factors  (not  necessariiv  covered  in  curricuium) 

9.  Safe,  decent,  and  beautiful  housing  (P3) 

Practice  Patterns/Approaches  to  Treatment 

1.  Does  the  suicide  risk  assessments;  often  refers  to  a  mentai  heaith  provider  (P2) 

2.  In-depth  discussions  of  treatment  options  with  patients.  (P10) 

3.  Prefers  to  ieave  medication  prescribing  to  a  speciaiist  in  situations  such  as  in 
patients  having  nightmares  (PI). 

4.  Hoiistic  approach  to  care  (P10) 

•  Identifying  patient’s  issues,  orientating  to  VA,  and  getting  patients  sieeping. 

5.  Uses  motivationai  interviewing  (Ml)  (PI,  P2,  P3,  P4,  P5,  P6,  P8,  P9,  P10) 

Q3 

Participants’  Ml  training  experience 

1.  Participated  in  Ml  training  (PI,  P2,  P5,  P9,  P10) 

•  Courses  with  VAMC  Heaith  Behavior  Coordinators  (PI,  P2) 

•  Center  of  Exceiience  in  Primary  Care  Education  (COE-PCE)  feiiowship 
training  (P2) 

•  Participates  in  various  Ml  trainings.  (P5) 

•  Basic  Ml  training  (P9,  P10) 

2.  Ml  expert  (P4,  P6,  P8) 

•  Teaches  Ml  as  part  of  the  COE-PCE  fellowship  program  (P4) 

•  Former  PI  on  an  Ml  training  project  for  PCPs.  (P6) 

•  Participated  and  helped  facilitate  Ml  trainings.  (P8) 

3.  No  formal  training  in  Ml  training.  (P3,  P7) 

4.  No  Ml  training  during  residency.  (P8) 

Ml  techniques  participants  find  useful 

1.  Open  ended  questions  (PI,  P5) 

2.  Repeating  back  to  veterans  what  s/he  is  hearing  (PI) 

3.  Exploring  patient’s  beliefs,  asking  to  explain  the  risks/benefits  of  a  particular 
behavior,  juxtaposing  patient  behavior  with  ideal  health  (P3,  P4,  P5) 

4.  Helping  veterans  set  their  own  goals  (P4) 

5.  Expressing  empathy  (P5) 

6.  Affirmations  (P5,  P8) 

7.  Readiness  Ruler  (P5,  P8) 

-Past  exposure  to  Ml  training  does  not  mean  that  the 
clinician  uses  a  range  of  Ml  techniques  or  have 
advanced  Ml  skills. 

-If  some  clinicians  believe  they  are  using  Ml  in  their 
practice,  the  audience  generation  strategies  may 
need  to  have  the  language  about  advancing  Ml  skills 
rather  than  introduction  fo  Ml. 

-Think  about  who  the  ideal  audience  is  for  the 
training?  Green  providers? 
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Q# 

Interview  Domain 

Implications 

•  P8  referred  to  this  as  a  confidence  ruier 

8.  Reflections  (P6,  P8,  P9) 

9.  Using  simpiicity  when  seiecting  techniques  to  use.  (P10) 

10.  Wouid  iike  to  bring  on  other  members  of  a  care  team  into  the  Ml  treatment  of 
veterans.  (P5) 

•  Participant  does  not  engage  in  this,  but  beiieves  it  wouid  be  a  great  idea. 

Challenaes  to  usina  Ml  in  clinical  settinas 

1 .  Relationship  with  Veteran  determines  whether  provider  wiii  use  Ml  during  interaction 
or  not.  (PI) 

2.  Knowing  what  Ml  ‘tool’  to  use  at  what  time.  (P9) 

Q4 

Shared  Decision-Makinq 

1 .  Variability  in  the  definition  of  shared  decision-making 

•  Appropriate  definition  (P2) 

•  Misconception  (PI) 

2.  SDM  is  time-consuming  as  it  requires  equal  presentation  of  options  (P2) 

3.  Lack  of  skills/confidence  necessary  for  SDM  (P4) 

-SDM  definition  may  need  to  be  provided  to  learners 
(as  additional  resource?) 

-Note:  A  question  about  SDM  was  not  asked  in  every 
interview.  (Our  focus  is  on  collaborating  with  patients 
on  a  personalized  care  plan.) 

Suqqestions — ^focus  on  content 

-Kaiser/community  docs  might  need  tips  for 
interacting  with  Vet  patients. 

-Easy  is  subjective  term.  How  do  we  determine  what 
is  easy  and  what  is  complicated  for  multiple 
participants? 

For  curriculum:  Be  sure  to  discuss  disrupted  fear 
networks  as  part  of  the  didactic 

1.  Include  different  kinds  of  trauma-childhood  trauma,  sexual  trauma,  abuse,  etc  (PI) 

2.  Make  the  training  easy  to  understand  and  not  complicated  (P2) 

3.  Include  information  on  how  fear  is  a  driving  force  for  illness;  trauma  lowers  threshold 
for  feeling  fear  (P3) 

4.  Include  psycho-social  aspects  of  illness.  (P3) 

5.  Include  how  to  express  gratitude  to  veterans  for  the  service  that  they  gave  to  the 
country.  (P3) 

6.  Make  sure  that  providers  understand  that  PTSD  treatment  is  unique  and  varies. 

(P5) 

7.  Address  provider  “trauma”  related  to  treating  patients  with  PTSD  (P3) 

Related  opinions 

1 .  Many  providers  don’t  have  an  outlet  to  de-compress  after  dealing  with  difficult  and 
emotionally  taxing/draining  patients.  (P8) 

DOMAIN  2:  TRAINING  DELIVERY:  FEASIBILITY  AND  ACCEPTABILITY 

Q6 

Experience  with  Virtual  World 

1.  No  experience  (P2,  P3,  P6,  P7,  P8,  P10) 

2.  Limited  experience,  no  similar  to  the  training  being  developed  in  this  project  (PI ,  P4, 
P5,  P9) 

-Providers  could  come  in  with  the  expectation  that 
they  know  what  to  do  in  the  environment,  but  find 
that  it’s  more  in-depth  than  they  originally  thought  or 
anticipated. 

Q7 

Acceptance  of  Virtual  World  Traininq 

1 .  Personally:  high  acceptance,  the  way  to  go  (PI ,  P2,  P6,  P7) 

•  Very  clever.  Could  be  additive  or  even  replace  current  training  modalities. 

-Our  recruitment/advertising  should  convey  precisely 
what  the  learner  can  expect  from  the  training.  VW  is 
still  unfamiliar  for  most. 
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Q# 

Interview  Domain 

Implications 

(P7) 

•  Has  potential  to  be  ‘incredibly  useful’  is  used  for  the  right  reasons.  (P9) 

•  Terrific  idea  with  the  potential  of  helping  providers  learn  how  to  really 
connect  with  veterans.  (P10) 

2.  Good  delivery  method  for  POP  (P1 ,  P2,  P6) 

•  Chance  for  a  higher  rate  of  attendance  because  it’s  online.  (P1) 

•  Convenient  -  at  own  desk,  at  your  environment,  etc.  (P1 ,  P2) 

•  Interesting/novel  way  of  learning  new  information;  good  delivery  method. 

(P2,  P6) 

•  May  be  beneficial  and  cost  effective  (P4) 

3.  Interactivity  between  users  is  a  positive  feature  (P4,  P6,  P7) 

•  You  may  forget  that  you’re  interacting  with  an  avatar  (P4) 

•  Values  simulated  patients  capability  (P7) 

4.  Face-to-face  training  may  be  converted  into  the  VW  training  (P10) 

5.  May  be  good  for  providers  who  are  new  to  VA  (P3) 

•  Mentions  Ukiah  as  a  clinic  with  providers  who  have  very  little  experience 
{P3) 

6.  Believes  it  will  be  well-received  by  younger  providers  (P1 ,  P4,  P5,  P9) 

•  Could  feel  too  artificial  to  older  generation 

7.  PCP  audience  generation  for  the  VW  training  will  be  a  challenge  (P1 ,  P2) 

8.  May  be  used  to  build  a  sense  of  community  among  providers.  (P8) 

9.  Skeptical/negative  (P2,  P3,  P5,  P8) 

•  Does  not  value  VW;  no  particular  benefit  to  this  sort  of  training  (P2,  P3,  P8) 

•  May  be  overwhelming  (P3) 

•  Training  modality  is  boring  (P5) 

Related  opinions 

1.  Necessary  change. (P4) 

2.  Curious  about  educational  impact/added  value  over  other  educational  strategies 
(P6,  P8) 

3.  Misconception  -  believes  that  participant  needs  virtual  reality  googles  to  participate 
in  training  (P6) 

4.  Values  PCP  learning  from  each  other  (P8) 

5.  Values  immediate  feedback  (P9) 

-  P4  described  in  her  own  words  what  we  call 
“immersive” 

-P4  may  be  invited  to  be  a  tester 
-Clearly  stated  technical  requirements  for 
participation  are  important 
-Benefits  to  synchronous  training: 

•  VW  may  support  building  community  among 
providers,  which  was  noted  to  be  important 
by  one  PCP.  PCPs  often  do  not  know  how 
their  peers  practice,  what  treatment  and 
communication  approaches  work  best  in  the 
practice  of  their  peers;  therefore,  PCPs  value 
opportunities  to  network  and  be  the  part  of 
the  community  where  sharing  of  experiences 
occurs.  The  VW  training  should  provide 
opportunities  for  learner-learner  interaction. 
(Think  about  opportunities  for  discussion, 
e.g.  homework.) 

•  Immediate  feedback  is  valued  -  this  theme 
supports  the  design  where  learners  practice 
new  skills  and  receive  feedback  from  the 
facilitators  and  other  learners. 

-PCPs  who  are  not  familiar  with  VW  may  be  skeptical 
about  education  in  VW.  Their  negative  attitude  may 
be  changed  once  they  engage  in  education  but  how 
to  make  them  choose  to  participate  in  the  training? 
-The  interviewed  PCPs  largely  represented 
older/mature  providers.  It  may  be  useful  to  have 
generation  mix  in  the  future  focus  group  to  receive 
feedback  from  younger  and  older  clinicians.  Also, 
can  potentially  use  their  “endorsement”  of  the  training 
as  a  recruitment  tool  for  the  RCT. 

-Note:  providers  who  are  new  to  VA  may  not 
necessarily  be  younger  providers;  staff  changes  may 
be  common  on  the  VA  facilities 

Suggestions — ^focus  on  delivery 

-Feedback  on  learner  performance  is  essential. 
-Perhaps  cite  benefits  of  VW  training  in  recruitment? 
-PCPs  rely  on  evidence-based  methods.  Can  we 
demonstrate  evidence  via  our  recruitment  tools? 

-Some  suggestions  are  universal  and  not  specific  to 

1.  Modules  in  18-20  minute  blocks  (PI) 

2.  Use  a  TED  talk  format  (PI ) 

3.  Make  the  training  user  friendly  (P2) 

4.  Make  the  training  similar  to  previous  trainings  participants  have  done  (P2) 

UNPUBLISHED  DATA  -  NOT  FOR  DISTRIBUTION 


22 


Q# 

Interview  Domain 

Implications 

5.  Provide  immediate  feedback  to  providers  iearning  new  Ml  skills  (P3) 

the  capabilities  of  VW,  such  as  case-based  learning. 

6.  Address  concept  of  forgiveness;  helping  the  soldiers  to  recognize  that  what  they  are 

role-playing  and  use  of  feedback.  The  evolving 

going  through  is  not  their  fault  (P3) 

curriculum  already  includes  these  elements. 

7.  Make  effort  to  present  training  as  ‘cool’,  ‘fun’,  ‘interesting’  and  ‘valuable’.  (P6,  P9) 

-Although  TED-style  presentation  is  an  interesting 

8.  Find  research  to  support  high  information  retention  rates  through  this  training 

format  to  explore  in  CME/CPD,  this  format  will  not  be 

modality.  (P6) 

the  part  of  this  curriculum 

9.  Avoid  ‘glitchy’  and  artificial  virtual  environment  (P6) 

10.  Make  the  training  efficient  for  providers  (P6) 

11.  Include  feedback  (P9) 

12.  Highlight  the  importance  of  the  training  (P9) 

13.  Should  not  be  formulaic.  (P9) 

14.  Make  training  short  (P10) 

15.  Make  CME  fun  and  engaging  (P10) 

16.  Use  case-based  learning  (P10) 

Related  opinions 

17.  Be  selective  with  the  initial  participants  to  increase  chance  of  getting  positive 
feedback (P6) 

18.  Role  playing  is  a  good  training  method  (P10) 

-VW  is  expected  to  be  fun  and  engaging 

DOMAIN  3:  BARRIERS  AND  FACILITATORS 

Q10 

Faciiitators  for  Participation  in  VW  training 

-Logistics  are  key.  Perhaps  we  should  speak  with  VA 

Q11 

1.  Having  blocked  clinic  time  (P1) 

training  experts  for  insight. 

2.  Having  dedicated  time  (P2,  P8,  P9) 

-It  is  important  to  create  a  safe  environment  to 

3.  Relevance  to  practice,  such  as  presenting  variety  of  causes  for  PTS  (PI) 

practice  skills 

4.  Free(P1) 

-Dedicated  or  even  blocked  clinic  time  seems  to  be 

5.  CME  credit  (P1 ,  P2,  P4,  P6,  PI 0) 

the  critical  factor.  If  potential  learners  have 

•  CME  credit  is  preferred  over  Ql  credit  (P4) 

designated  days/times  for  education  in  their  setting. 

•  Nice  but  not  a  big  draw  (P6) 

the  planners  need  to  be  aware  of  these  scheduled 

6.  Ql  credit/incentive  that  contributes  to  re-certification  (P2,  P6) 

times  and,  if  possible,  schedule  VW  training  sessions 

7.  “Self-paced”  training  (P2) 

accordingly. 

8.  Being  able  to  do  training  from  the  VA  or  at  home  (P2) 

-CME  credit  may  be  a  desired  feature  of  the  training 

9.  Allowing  for  both  desktop  and  mobile  access  (P2) 

but  it  is  not  likely  to  be  the  major  factor  influencing 

10.  Gift  card  (P6) 

participation 

1 1 .  Purchasing  VR  googles  for  providers  (P3) 

-Ql  theme  may  reflect  VA-specific  Ql  requirements 

12.  Monetary  incentive  and  credit  may  not  mean  as  much  as  the  opportunity  to  learn 

rather  than  MOC  requirements 

something  new  (P4) 

-Collecting  testimonials  from  VW  training  participants 

13.  “Mental”  age  of  the  provider  (P4) 

may  be  helpful  for  generation  of  future  audience  for 

14.  Non-judgmental  environment  (P4) 

this  training 

15.  Novelty,  technology  coolness  factor  (P6) 

-Training  description/front  matter  may  have  a 

16.  Easy/user-friendly/fun  to  use  (P4) 

reference  to  the  effectiveness  of  education  in  VW. 
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Q# 

Interview  Domain 

Implications 

17.  Positive  feedback  from  participants  (P4,  P6) 

•  Word  of  mouth  advertising  among  providers  (P4) 

•  Testimoniais  (P6) 

18.  Food  (P6) 

19.  Demonstrate  positive  educationai  impact  (P8) 

20.  Added  patient  vaiue  (?)  (P9) 

Q9 

Barriers  to  Participation  in  VW  training 

-Revisit  IT  logistics. 

1.  Resistance  because  it  is  new  and  different  (PI) 

2.  Finding  time  for  the  training  (P2,  P3,  P5,  P6) 

3.  Time  to  create  an  account  and  iearn  how  to  use  the  program  (P5) 

4.  Fear  of  technoiogy  (P4,  P6) 

5.  Computer  probiems  (PI,  P2) 

•  Not  being  abie  to  access  the  training  from  a  VA  computer  (P2) 

•  Not  being  abie  to  come  back  to  finish  the  training  (P2) 

6.  Having  to  pay  for  the  training  (P2) 

7.  Cider  age  (P4) 

-VA  access  to  the  training  is  critical 

DOMAIN  4:  GENERALIZABILITY:  APPLICABILITY  TO  BROADER  AUDIENCE 

Q12 

Suggestions  for  how  to  make  training  applicable  to  broader  audience 

-It’s  important  to  have  a  sample  of  learners  that’s 

1.  Include  childhood  trauma,  sexual  trauma,  abuse,  etc.  (PI) 

diverse  in  age,  training,  geo.,  etc. 

2.  Reflect  a  provider’s  most  common  type  of  patient  (P2) 

-Training  should  at  least  present  co-morbid  MH 

3.  Reflect  PTS  with  comorbid  MH  issues  (P2) 

issues,  if  not  discuss  determining  if  the  patient  is 

4.  Include  range  of  patients  in  age,  experience,  and  treatment  options  (P2) 

facing  PTSD  vs.  other  MH  issues. 

5.  Have  profession-specific  training  modules.  (P2,  P4) 

-  Question:  Will  learners  from  outside  VA  have  the 

6.  Use  stronger  form  of  training  than  ‘passive  learning’.  (P4) 

same  exposure  to  patients  with  PTSD? 

7.  Vary  participant’s  age  ranges  for  the  initial  testing  group  to  create  ambassadors  for 

-  Necessary  to  address  how  different  members  of  a 

the  program.  (P4) 

treatment  team  (varied  profession)  would  approach 

8.  Keep  the  material  down  to  a  bare,  irreducible,  minimum  (BIM)  when  creating 

this?  Interprofessional/collaborative  practice 

content,  then,  build  off  the  BIM  (P9) 

9.  Keep  the  training  relevant  with  current  events  in  the  medical  world  (zika,  infectious 

emphasis. 

disease,  etc.)  (P4,  P7) 

-We  should  pay  attention  to  age  during 

•  Providers  already  have  to  learn  about  these  things  and  this  could  make  this 
modality  more  appealing  to  them.  (P7) 

10.  Be  sensitive  to  the  broad  range  of  care  issues  that  PCP  have  to  deal  with  each  day 
(P4) 

11.  Show  educational  impact  (P6) 

Other  opinions 

1 .  May  not  be  generalizable  the  VA  experience  is  different  from  the  experience  in  the 
non-VA  settings  (P2) 

randomization  process  for  focus  groups. 
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Appendix  B:  Virtual  Environment  Images 


The  figures  in  this  section  represent  the  deiiverabies  submitted  by  the  vendors,  consistent  with  the  SOW. 
CNDG  provided  the  foiiowing  disciaimer:  “This  is  a  proof  of  concept  for  NOIRE.  Finai  program  may  be 
somewhat  different.” 

Figure  1.  Virtuai  campus  (bird’s  eye  view) 


Figure  2.  Virtuai  Campus 


Figure  4.  Orientation  Center  displaying  PCP  avatar 


Figure  5.  Virtual  objects:  classroom  aids 


Figure  7.  PCP  avatars  observing  the  Standardized  Patient  in  an  exam  room 


Appendix  C:  Overall  Training  Storyboard 
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PTSD  Project 

Overall  Notes 
9.23.16 


Improving  Access  to  Care  for  Warfighters 
Virtual  World  Curriculum  Overview 
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Overall  Look  and  Feel  Summary 

•  Location  to  be  reminiscent  of  the  SF  VA  facility  with 
view  of  the  channel  into  the  SF  Bay,  greenery,  etc. 

•  Open-air  amphitheater,  automatic  seating  for  25 
avatars,  stage  area  with  screen  (for  slides)  and  a  back 
screen  for  speakers'  reference. 

•  Nearby  building  that  resembles  (inspired  by?)  the  SF 
VA  hospital  (a  large,  art  deco,  stucco  building)  where 
the  Exploratoriums  and  the  small  groups  will  take 
place)  -  see  photos. 

•  Will  need  some  official  landing  place  where  everyone 
enters. 
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Exploratorium  1  and  2 

•  Exploratoriums  1  &  2  are  located  in  two  separate  areas 
of  our  hospital  lobby. 

•  The  lobby  should  look  like  a  modern  hospital  lobby 
(you  can  tell  you're  in  a  hospital  but  it's  nice  -  plants, 
natural  light,  couches). 

•  Exploratorium  1  will  be  available  to  them  upon  arrival 
for  session  1  and  will  remain  available.  Exploratorium 
2  will  only  be  available  upon  arrival  for  session  2. 

•  The  two  Exploratoria  will  consist  of  numbered  stations 
that  the  learners  will  explore  in  numerical  order  as  a 
group.  They  can  return,  asynchronously,  to  revisit. 


4 


10/24/2016 


Amphitheater 

•  Both  sessions  will  include  a  didactic  portion  to 
take  place  in  a  spacious  and  attractive  open-air 
amphitheater. 

•  Automatic  seating  for  25  avatars 

•  Screen  for  PPT  slides 

•  Reference  screen  for  speakers 

•  Teleporters  easily  available  to  access  the  small 
group  breakout  rooms 

•  Possible  display  of  completed  "boards"  from  the 
Exploratorium  activities  in  the  amphitheater 
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Small  Group  Rooms 


•  We  will  have  18  learners  at  each  session,  broken  into  3  small  groups  of  6 
learners  each  for  the  practicum  portion  of  the  curriculum. 

•  After  the  didactic  session,  the  learners  will  be  teleported  to  their  rooms 
(all  within  the  "hospital")  for  the  role-play  practice  to  take  place  in  their 
small  groups. 

•  Rooms  should  be  identified  by  some  easy  reference  -  possibly  color  (blue 
room,  green  room,  etc).  We  will  pre-assign  avatars  to  groups/  rooms. 

•  Each  small  group  room  will  be  an  examination  room  (familiar  to  our 
learners)  that  will  include  two  obvious  chairs  where  the  standardized 
patient  and  the  learner  will  sit  (one  by  one)  to  perform  their  role  play.  The 
other  learners  in  the  group  will  also  have  seating  with  a  clear  view  of  the 
examining  room  where  the  role  plays  take  place. 

•  Sitting  in  all  chairs  should  be  automatic,  on  click. 

•  Place  an  obvious  looking  timer  for  the  Small  Group  coach  to  keep  track  of 
time  (8  -  10  minutes  per  role  play). 

•  Large  screen  visible  that  includes  the  facts  of  Alex's  case  that  everyone  will 
be  using. 
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Avatars 

•  Will  need  18  avatars  for  the  learners,  all  looking 
professional  with  business  casual  attire.  Three  cohorts 
don't  overlap  so  these  avatars  can  be  re-used. 

•  Mixture  of  male  and  female,  different  skin  tones,  ages, 
weight.  Women  should  not  be  over-sexualized. 
Provide  a  healthy  mix  so  the  audience  looks  diverse. 

•  Also  avatars  for  the  presenter,  the  3  small  group 
coaches,  and  ~6  avatars  for  staff. 

•  Presenter  and  coach  avatars  should  have  built  in 
animation  actions  (typical  speaker  movements). 
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Homework 

•  There  will  be  homework  for  the  learners 
between  Session  1  and  Session  2. 

•  All  small  group  interviews  will  be  video 
recorded  and  posted  (TBD)  for  review. 

•  Learners  are  asked  to  review  their  own 
recording  and  those  of  2-3  others  and 
comment  on  them. 

•  Facilitator  will  debrief  the  homework  at  the 
start  of  Session  2  didactic  section. 
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Appendix  D:  Session  One  Storyboard 
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Session  1  Exploratorium: 
An  Immersive  Introduction 

9.23.16  Draft 

Learners  will  spend  30  minutes  in  the 
Session  1  Exploratorium 
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Learning  Stations 

1.  Meet  Alex  with  Introduction  to  Motivational 
Interviewing 

2.  PTSD  Symptomology 

3.  What  is  PTSD? 

4.  Alex's  Biomonitor 


1.  Meet  Alex 

•  "Alex"  will  be  the  case  that  runs  through  both  sessions. 
Here  the  learners  meet  Alex  and  get  to  know  his  situation. 

•  The  learners  enter  an  examination  room  -  a  scene  with 
Alex  sitting  on  an  examination  table,  a  doctor  in  a  lab  coat, 
a  large  computer  screen  showing  his  medical  chart  with 
facts  visible,  a  video  monitor,  and  an  Ml  poster  on  the  wall. 

•  A  video  launches  of  a  doctor/patient  conversation.  This  is 
the  first  of  two  videos  (aka  "standard  interview"),  ending 
with  doc  wishing  she  could  learn  more. 

•  After  the  first  video,  a  second  video  launches  with  better 
results. 

•  At  the  conclusion  of  the  2'^'^  video,  the  door  to  Alex's 
apartment  opens. 

•  Learners  are  instructed  to  enter  the  door  on  the  right  to 
explore  Alex's  home. 
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Alex's  Medical  Chart 


38  YO  Iraq  Vet  here  for  follow-up 

Active  Issues 

Medications 

Weight  gain 

Chronic  low  back  pain 

Disordered  sleep 

Poor  memory  and  concentration 

Anxiety  and  Depression 

Hypertension 

•  Percoset  5/325  1-2  tabs  po  q8  prn 

•  HCTZ  25  mg  po  daily 

Vital  Signs 

Appointment  History 

•  Age:  38 

•  Weight:210 

•  BMI:29 

•  BP:  145/94 

•  Primary  Care:  2  months  ago 

•  Mental  Health:  NO  SHOW 

•  Physical  Therapy:  NO  SHOW 

Service  History 

•  Operation  Iraqi  Freedom 

•  Completed  two  deployments 

•  Ended  service  2  vears  aso 

Motivational  Interviewing 
Summary  Poster 

-  Express  empathy  (non-judgmental  approach) 

-  Develop  discrepancy;  encourage  change  talk 

-  Support  patient's  self-efficacy 


3 


10/24/2016 


Video  #1  Production  Notes 

•  PCP  is  female,  dressed  in  a  blazer,  wearing  stethoscope, 
has  glasses 

•  Characteristics 

-  Judgmental  tone,  fast  talking,  doesn't  wait  for  a  response 
from  Alex  before  moving  on  to  the  next  question 

•  PCP  body  language 

-  Looking  at  medical  chart ,  not  necessarily  at  the  patient 

-  She  looks  over  her  glasses  at  Alex  in  a  patronizing  way 

•  Alex's  body  language 

-  Very  stiff,  unchanging,  arms  crossed,  clearly  not  engaged 


Video  #1:  Interview  Script 

K:  Hello  Alex.  How's  it  going?  How  can  I  help  you  today? 

A:  Fine. 

K:  So,  looks  like  you've  put  on  some  weight,  even  since  last  time-  didn't  we  go  over  a  diet  and  exercise 
plan?  And  my  goodness,  guess  the  blood  pressure  meds  aren't  working  so  well. ..your  blood  pressure  is  still 
high.  Are  you  taking  the  medication? 

A:  Well  uh... sometimes  I  forget  and  forgot  this  morning. 

K:  And  I  got  a  message  from  my  nurse  that  you're  requesting  your  Percoset  a  week  early... Is  there  something 
going  on  that  you  want  to  talk  to  me  about? 

A:  No,  not  really,  just  the  pain  is  worse;  can't  get  to  the  gym  and  school  is  NOT  going  well. 

K:  Sorry  to  hear  that  Alex.  Is  there  some  way  I  can  be  of  help?  Do  you  want  to  talk  about  it?  Do  you  want  to  see 
a  colleague  of  mine  in  mental  health?  I  know  you  didn't  show  for  your  last  mental  health  appointment,  but  I 
can  re-refer  you.  And  what  about  physical  therapy  for  your  pain? 

A:  Well  not  really;  I  talk  to  my  wife  and  the  PT  doesn't  help  my  pain.  The  meds  do. 

K:  You  mean  the  Percoset? 

A:  Yeah  because  if  I  don't  have  that  I'm  gonna  drink  more  to  deal  with  the  pain. 

K:  The  pain?  You  mean  your  back  pain  or  the  thoughts  about  what  happened  to  you  and  your  buddies  in  Iraq.... 
A:  All  that's  in  the  past.  I  don't  want  to  talk  about  it. 
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Video  #2:  Production  Notes 


•  Same  PCP  (female,  dressed  in  a  blazer,  wearing 

stethoscope,  has  glasses) 

•  PCP  body  language 

•  Faces  Alex  and  engages  him.  "Open"  affectation. 
Nods  with  understanding. 

•  Eyeglasses:  will  take  off  glasses  and  fully  engage  in 
eye  contact  while  speaking  with  Alex. 

•  Alex's  body  language 

•  Starts  with  arms  crossed  and  gaze  averted.  He 
then  gradually  eases,  opens  his  arms,  makes  eye 
contact,  sits  forward 


Video  #2:  Interview 


K:  Hello  Alex.  It's  been  awhile.  How  are  you  today? 

A:  Fine. 

K:  What's  going  on  for  you? 

A:  Just  want  my  meds-  my  back  pain  is  worse  and  I'm  drinking  behind  it  cuz  I  can't  get  enough  meds. 

K:  Sounds  really  rough.  Tell  me  more  about  that. 

A:  Look- 1  need  to  get  home  to  take  care  of  my  kid-  wife's  got  to  work.  I'm  trying  to  hang  in  at  school  so  I  can  get  a 
job.  I  just  don't  have  time  for  this  right  now. 

K:  Sounds  like  you  have  a  lot  of  pressure  on  you  and  I  know  you've  been  through  quite  a  lot. 

A:  Yeah  -  it's  rough.  And  I  have  these  thoughts-  like  I  am  just  right  back  there  and  there's  nothing  I  can  do  to  stop 
it-  it  just  happened-  boom-  and  them  he  was  gone.  My  buddy  was  gone.  But  I'm  ok. 

K:  You're  ok? 

A:  Yeah...  I  am  {slow),  but  when  I  get  these  thoughts  my  pain  gets  going  in  my  back.  I  need  more  medicine,  but 
then  that's  not  enough,  so  I  start  to  drink.  I'm  getting  so  sick  of  this  cycle  -  like  I  really  need  some  help  doc,  but  I 
don't  have  time;  I  don't  want  to  go  over  it  again  with  someone  new. 

K:  Alex,  what  I'm  hearing  is  that  on  the  one  hand  you  don't  have  the  time  or  don't  want  to  look  at  it,  but  on  the 
other  hand  you  really  care  about  your  wife  and  child  and  want  to  graduate  and  get  a  job. 

A;  Yeah.  My  wife  and  kid  are  everything  to  me  right  now  and  I  want  to  be  there  for  them. 

K:  I  hear  you.  You  know,  becoming  more  aware  of  what's  really  going  on  is  Step#l.  You  are  well  on  your  way. 

Blue  =  Expressing  Empathy 
Purple  =  Developing  Discrepancy 
Green  =  Supporting  Self-Efficacy 
Red=  Change  Talk 
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Enter  Alex's  Apartment 

•  As  the  second  (Ml -infused)  video  concludes, 
the  door  to  Alex's  home  opens  (as  if  the 
conversation  has  "unlocked"  our  view  into 
Alex's  world) 

•  Learners  enter  Alex's  home 

•  Items  to  observe:  see  next  slide 


Item 

Significance 

Living  room  couch  is  made  up  as  a  bed 

Indicates  sleep  disruption 

Toddler  in  a  play  pen,  off  to  the  side,  rotating 
through  series  of  upset  motions  and  wife 
standing,  rotating  through  a  series  of  gestures 
indicating  her  frustration 

Indicate  disconnect  from  family 

Framed  picture  of  Alex  and  wife  hugging  upon 
his  arrival  home  (classic  reunion  shot) 

Value  of  family  to  Alex 

#1  Dad  coffee  mug  or  trophy 

Value  of  family  to  Alex 

Prescription  pill  bottles  (2)  on  the  table 

Opioid  dependence,  chronic  back  pain 

Recycling  bin  with  many  beer  cans 

Alcohol  dependence 

Bottle  of  whiskey  next  to  the  pill  bottles  on  the 
table 

Alcohol  dependence 

Legacy  running  shoes  and  fishing  pole  visible  in 
a  nearby  closet 

Former  hobbies  and  interests 

A  collection  of  framed  pictures  on  the  wall  - 

Used  to  run,  used  to  fish  -  these  activities 

one  large  one  of  Alex  &  wife  -  including  a 
photo  of  Alex  in  running  gear  crossing  a  finish 
line  and  another  of  Alex  holding  up  a  just- 
caught  fish 

were  clearly  important  to  him  at  one  time. 

School  books  open  on  the  table 

Trouble  with  concentration 
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Transition: 

Summarize  Observations 
About  Alex's  Apartment 

•  The  learner  is  guided  to  observe  the 
important  (tell-tale)  items,  as  they  walk 
around  the  room. 

•  After  they  exit  the  apartment,  learners  are 
asked  to:  "Name  two  observations  in  Alex's 
apartment  which  will  help  with  your 
assessment." 


2.  PTSD  Symptoms 

•  Learners  move  to  station  #2  to  find  out  more 
about  PTSD  symptoms 

•  A  circular  screen  emerges  from  the  floor, 
surrounding  the  learners. 

•  Projected  on  the  screen,  a  series  of  still  images 
accompanied  by  audio  of  Alex's  voice,  making 
statements  about  his  situation.  The  still  images 
depict  the  facts  that  Alex  describes.  Words 
appear  on  the  screen  that  categorize  the 
symptoms  he  describes.  The  cycle  repeats. 
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Alex's  Symptoms 


1  What  Alex  says 

Symptom  Category 

Image 

There  was  an  lED  explosion  -  that  was  very 
close,  but  I'm  ok. 

Traumatic  event 

lED  explosion  near  a  tank  in  Fallujah 

When  1  was  sitting  in  class  or  trying  to  do 
homework,  1  kept  thinking  about  what 
happened  in  Iraq.  1  also  have  some  crazy 
dreams  about  it. 

Intrusions/Re-experiencing 

Young  man,  representing  Alex,  sitting 
in  a  classroom,  trying  to  concentrate. 
Person  thrashing  around  in  bed, 
having  a  nightmare. 

I'd  really  rather  not  talk  about  it-  best  to  just 
put  it  to  rest. 

Avoidance 

Young  man,  representing  Alex,  head  in 
hands,  not  communicating 

When  1  think  about  what  happened,  it  really 
bums  me  out.  1  feel  like  it  didn't  have  to 
happen. 

Negative  alterations  in 
cognition  and  mood 

Our  young  man  looking  lost,  sad, 
depressed. 

It  was  hard  for  me  to  be  in  a  crowded 
classroom.  My  heart  started  racing  and  1 
just  needed  to  get  out. 

Alterations  in  arousal  and 
reactivity 

Our  young  man  in  a  crowded  scene  - 
room  full  of  people. 

My  wife  isn't  too  happy  with  me  lately  and 
that's  why  I'm  here  today.  1  also  lose  my 
temper  with  my  kid  sometimes  and  that 
makes  me  feel  bad. 

Negative  impact  on  functioning 
(Employment,  parenting, 
relationships) 

Alex's  wife,  looking  angry,  fed-up, 
hands  on  hips. 

15 
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Transition 

•  After  the  images  and  audio  finish,  the 
circular  screen  from  station  #2  lowers 
back  into  the  floor  and  Alex's  voice  says, 
"What  is  going  on  with  me?" 


3.  What  is  PTSD? 

•  Alex's  question  leads  them  to  the 
start  of  the  next  learning  station  - 
a  simple  poster  titled  'What  is 
PTSD?' 

PTSD  (Posttraumatic  Stress  Disorder)  is  a 
mental  health  problem  that  some  people 
develop  after  experiencing  or  witnessing  a 
life-threatening  event,  like  combat,  a  natural 
disaster,  a  car  accident,  or  sexual  assault. 
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Further  Explanation 

•  A  second  poster  explains: 

•  With  time,  good  self-care,  and  in  some  cases, 
treatment  or  counseling,  most  people  recover 
completely  from  trauma. 

•  In  some  individuals  the  symptoms  following 
trauma  do  not  resolve  or  get  worse,  can  last  for 
months  or  even  years  and  can  interfere  with 
daily  functioning. 

•  Let's  take  a  look  at  the  biology  behind  this... 


The  Biology  of  PTSD 

•  The  learners  move  on  to  examine  the  biology  behind  the  condition 

•  The  path  takes  them  to  a  3D  image  of  the  brain  with 
"Parasympathetic  Division"  on  one  side  and  "Sympathetic  Division" 
on  the  other.  They  are  instructed  to  click  on  Sympathetic  first. 

•  When  they  click  on  Sympathetic,  they  see  a  signal  start  in  the  brain, 
travel  down  the  spinal  column,  hitting  nerve  centers  along  the  way, 
and  the  information  displays. 

•  At  signal's  end  the  words  come  up  "Fight  or  Flight  Response" 

•  When  they  click  on  Parasympathetic,  a  signal  starts  in  the  brain  and 
goes  just  to  the  brain  stem,  and  the  information  displays. 

•  At  signal's  end  the  words  come  up  "Rest  and  Digest  Response" 
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Parasympathetic  division 


Constricts  pupil  of  eye 


Stimulates  salivary 
gland  secretion 

Constricts 
bronchi  in  lungs 

Slows  heart 


Stimulates  activity  of  >\ 

;  xV 


stomach  and  intestines  : 


Stimulates  activity 
of  pancreas 


Stimulates  gallbladder 


■>->  Thoracic  -< 


Lumbar  - 


Sympathetic  division 
<  Dilates  pupil  of  eye 


Inhibits  salivary 
gland  secretion 

Relaxes  bronchi  in  lungs 

Accelerates  heart 

Inhibits  activity  of 
stomach  and  Intestines 


Inhibits  activity 
"A  ;  of  pancreas 


Promotes  emptying 

of  bladder  >— >— 

Promotes  erection  .  ^ 
of  genitalia 


Sacral 


Synapse 


Stimulates  glucose 
-<  release  from  liver; 
inhibits  gallbladder 

Stimulates 
adrenal  medulla 

.  Inhibits  emptying 
of  bladder 


Promotes  ejaculation  and 
^  vaginal  contractions 


C  Pearson  Education,  Irtc. 

•  »14  rwngn  I  duoWK  nc 


Voice  Over 

•  Experiencing  fear  during  a  trauma  is  a  normal  and  adaptive 
response.  Fear  activates  the  sympathetic  division  of  the 
nervous  system  which  releases  epinephrine  and 
norepinephrine  that  act  on  different  organs  in  the  ways  you 
see  here.  This  is  termed  the  "fight-or-flight"  response. 

•  Frequent  repeated  arousal  of  the  sympathetic  response  can 
lead  to  chronic  stress-  related  health  problems  such  as 
hypertension,  obesity,  cardiovascular  disease  and  diabetes. 

•  Patients  suffering  from  PTSD  become  fearful  not  only  of  the 
trauma  itself,  but  of  their  reactions  to  the  trauma.  Body 
signals  that  were  once  providers  of  key  information  become 
"dangerous".  In  this  way  it  becomes  a  terribly  vicious  circle, 
reinforcing  itself. 
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Comorbidities 

PTSD  is  complex  because  it  rarely  exists  on  it's 
own.  Individuals  with  PTSD  typically  have  other 
comorbid  mental  health  problems,  the  most 
common  of  which  is  depression.  PTSD  can  also 
co-occur  with  anxiety,  panic,  traumatic  brain 
injury  (TBI)  and  substance  used  disorders,  most 
commonly  alcohol  abuse/dependence,  but  also 
opioid  abuse/dependence  and  other  drugs, 
including  tobacco. 


Statistics 

•  As  the  learners  leave  this  third 
station,  there  will  be  a  final  display  of 
statistics  of  the  disease  for  them  to 
wander  through. 
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PTSD  Statistics 

-About  10  of  every  100  (or  10%)  of  women 
develop  PTSD  sometime  in  their  lives 
compared  with  about  4  of  every  100  (or 
4%)  of  men. 

-About  11-20  out  of  every  100  Veterans  (or 
between  11-20%)  who  served  in  Iraq  and 
Afghanistan  have  PTSD  in  a  given  year; 
more  than  30%  of  Iraq  and  Afghanistan 
presenting  to  VA  have  received  a  diagnosis 
of  PTSD. 


4.  Alex's  Biomonitor 

•  Learners  move  to  the  fourth  and  final  station  in 
Exploratorium  1. 

•  Show  a  video  with  a  companion  vital  signs  monitor 
(see  slide  for  suggested  layout).  Watch  video  and 
accompanying  monitor  changes  in  vital  signs. 

•  Video  1  shows  a  soldier,  Alex,  driving  tank  through 
Fallujah  streets,  with  extreme  adrenergic  response 
(demonstrated  through  increase  in  blood  pressure, 
heart  rate,  and  respiratory  rate) 

•  Video  2  shows  Alex  driving  a  car  on  a  California 
highway  with  normal  vital  signs.  Trigger  event  (car  in 
front  of  him  back-firing)  and  his  vital  signs  shoot  up. 

•  CA  video  morphs  into  Fallujah  scene  at  the  end. 


15 


10/24/2016 


Physiology  Values:  Video  #1  (Fallujah) 


Resting  vitals: 

-  BP:  136/85  (slightly 
stressed  from  driving 
tank) 

-  RR:  14  breaths  per 
min 

-  Pulse:  85  BPM 


Activated: 

-  BP:  180/100 

-  RR:  22  breaths  per 
min 

-  Pulse:  127  BPM 
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Physiology  Values:  Video  #2 

(CA  Highway  1) 

•  Resting  vitals: 

-  BP:  180/100 

-  BP:  129/80 

-  RR:  22  breaths  per 

-  RR:  12  breaths  per 

min 

min 

-  Pulse:  127  BPM 

-  Pulse:  75  BPM 

•  Activated: 

End  of  Exploratorium  1 
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Session  2 

Exploratorium;  An  Immersive 
Introduction 

9.23,16  draft 

Learners  will  spend  25  minutes 
exploring  the  Session  2  Exploratorium. 
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Learning  Stations 

1.  Barriers  to  Care 

2.  Clarifying  Your  Patient's  Values 

3.  The  Four  Care  Modalities 

1.  Behavioral  Health 

2.  Mind-Body/Integrative 

3.  Medication 

4.  Community  Engagement/Psychosocial  Approach 

4.  Personalized  Care  Planning  using  SMART 
Goals 


1.  Barriers  to  Care 

•  Learners  will  enter  the  session  2 
Exploratorium  through  a  barriers-to-care 
jungle.  As  they  walk  along  a  green,  leafy  path, 
various  representative  'barriers  to  care'  will 
sprout  up  out  of  the  ground,  waft  by  them, 
dive-bomb  from  above,  seeming  to  block  their 
way  to  the  next  learning  station. 
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Sample  Barriers  to  Care 

•  Not  enough  time  with  each  patient 

•  I  don't  know  when  to  refer 

•  I  am  not  a  mental  health  practitioner 

•  My  patient  has  a  'warrior  mentality',  just  tough  it 
out 

•  Stigma  around  PTSD 

•  My  patient  is  in  denial 

•  I  am  not  a  mental  health  practitioner 

•  Screening  tools  aren't  working  for  me 

•  Live  too  far  from  services 

•  My  vet  patients  are  avoidant 


2.  Clarifying  Your  Patient's  Values 
What  Really  Matters  to  Alex? 

*  Coming  out  of  the  Barriers  jungle,  learners  arrive  in  the 
Care  Planning  Board  construction  area. 

*  Welcoming  open  space  with  a  blank  Care  Planning  Board 
for  each  participant  ringing  the  room. 

*  Learners  are  asked  to  remind  themselves  of  Alex's  values. 

-  Reflecting  back  on  Alex's  home  environment,  which  you 
saw  in  session  1,  what  do  we  know  about  him?  His 
values?  What  matters  most  to  Alex? 

-  What  are  Alex's  symptoms? 

*  Find  your  Care  Planning  Board  and  type  what  you 
remember  about  Alex  into  the  first  and  second  quadrants. 
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PCP  AVATAR  NAME 


Alex’s  Values 

Alex's  Symptoms 

Alex's  Treatment  Options 

SMART  Goal 

What  matters  most? 

Learners  could  revisit  Alex's  apartment  to  gather  observations... 


Item 

Significance 

Framed  picture  of  Alex  and  wife  hugging 
upon  his  arrival  home  (classic  reunion 
shot) 

Family  matters 

#1  Dad  coffee  mug  or  trophy 

Family  matters,  specifically  being  a  father. 

Legacy  running  shoes  and  fishing  pole 
visible  in  a  nearby  closet 

Being  fit  and  having  hobbies 

A  collection  of  framed  pictures  on  the 
wall  -  one  large  one  of  Alex  &  wife  - 
including  a  photo  of  Alex  in  running  gear 
crossing  a  finish  line  and  another  of  Alex 
holding  up  a  just-caught  fish 

Used  to  run,  used  to  fish  -  these  activities 
were  clearly  Important  to  him  at  one 
time. 

School  books  open  on  the  table 

Working  toward  a  better  future;  getting 
an  education;  getting  a  job 
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3.  Four  Care  Modalities 

•  Once  the  first  quadrant  of  the  Care  Planning  Board  is  filled 
in  for  all,  the  transport  vehicle  arrives. 

•  Hop  aboard  the  transport  vehicle  and  travel  to  the  four 
modality  display  areas 

•  Each  display  area  includes  one  diorama  for  each  treatment 
option 

*  Learners  select  what  they  think  will  be  good  treatment 
options  for  Alex,  based  on  his  values 

*  Their  selections  are  captured  on  some  kind  of  a  PDA/HUD 

*  Return  to  Care  Planning  Board  construction  area,  "upload" 
your  selections  to  your  own  Care  Planning  Board 


Multi-Modal  PTSD  Management 


•  Cognitive  behavioral  therapy 
or  other  evidence-based 
psychotherapies,  i.e.,  stress 
innoculation  therapy  (SIT) 

•  Online  or  mobile  app 
support 

•  Substance  use  treatment  i 


SSRIsorSNRIs 

Prazosin 

Sleep  medication 
(e.g.  trazadone 


'  Behavioral 
'  Health 


Medication 


/ID. 


Mind-Body/ 

Integrative 


•  Aerobic  exercise 

•  Acupuncture 

•  Mindfulness/Meditation 

•  Tai  Chi  or  yoga* 


j  Community 
/  Engagement/ 

!  Psychosocial 

Approach 


Vocational  rehab 
Volunteering 
Hobbies 
Support  groups 
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Behavioral  Health 


Intervention 

How  to  Depict  in  a  Diorama 

Cognitive  Behavioral  Therapy  and 

Other  Evidenced-Based 

Psychotherapies 

Two  people  facing  each  other  in  chairs.  One  is  a 
doctor  and  has  a  notepad  and  the  other  is  Alex 

Online  or  mobile  app  support 

Smart  phone  with  apps  showing  (e.g.  PTSD  Coach, 
afterdeployment.org) 

Substance  Use  Treatment 

Semi-circle  of  chairs  with  a  whiteboard  and  an 
obvious  facilitator  writing  words  such  as  “craving” 
and  “breathe”  (group  therapy) 

Medication 


Intervention 

How  to  Depict  in  a  Diorama 

SSRIs  and  Antidepressants 

Bottle  of  SSRIs  (with  readable  Rx) 

Prazosin 

Bottle  of  Prazosin 

Sleep  medication 

Bottle  ofNon-benzo  Sleep  Aid 
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Mind-Body/Integrative 


Intervention 

How  to  Depict  in  a  Diorama 

Aerobic  exercise 

Alex  wearing  a  pair  of  running  shoes  and  mnning 
shorts. 

Acupuncture 

Alex  laying  face  down  on  table  with  acupuncture 
needles  in  place  along  his  spine. 

Meditation 

Alex  seated  on  ground,  on  a  zafii;  straight-backed 
position,  eyes  closed. 

Tai  Chi  or  Yoga 

Alex  in  a  downward-dog  position,  on  a  yoga  mat 

Community  Engagement/ 
Psychosocial  Approach 


Intervention 

How  to  Depict  in  a  Diorama 

Vocational  Rehab  (job  training) 

Alex  seated  at  a  computer  with  someone  beside  him 
implying  teaching.  Screen  can  say  “HTML  Basics”. 

Hobbies 

Alex  with  a  fishing  pole 

Support  Groups 

Full  circle  of  people  sitting  in  chairs  (including  Alex) 
looking  at  one  another  and  listening  attentively. 

Volunteering 

Alex  wearing  a  T-shirt  that  says  “Coach”  next  to  a 
track  with  some  little  kids. 
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Transition  from  #3  to  #4 

•  Once  the  learners  download  their  choices  to 
their  Care  Planning  Board,  they  move  to 
learning  station  #4 


4.  SMART  Goals  for  Personalized 
Care  Planning 

•  Large  poster  board  with  the  definition  of  a 
SMART  goal  (see  next  slide) 

•  Another  display  with  Alex's  original  medical 
chart  (reminder) 

•  Learner  is  presented  with  a  series  of  sample 
patient-constructed  goals 

•  For  each  sample  goal,  they  vote  "yes"  or  "no" 
(is  it  a  SMART  goal  or  not?) 

•  Feedback  presented  based  on  your  selection 
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What  is  a  SMART  Goal? 


What  is  a  goal? 

•  Something  patients  are  n^  currently  doing 

•  Something  that  is  a  reach  or  a  stretch  for  them 


Smart  Goals 

•Specific 

»Measureable 

»Attainable 

« Results  Focused  ~ 

•Time-Focused 


Sample  Un-SMART  Goal  A 

•  "I'm  going  to  volunteer  at  the  Boys  &  Girls  Club  to 
get  out  of  the  house  and  help  others." 

•  Feedback:  This  is  not  a  specific  goal  and  it's 
vague  (so  tough  to  measure).  Without  a  specific 
plan  or  a  regular  schedule,  will  be  hard  to  initiate, 
stick  to  and  measure  progress. 

•  Positives:  This  is  a  good  start,  because  it's 
something  patient  and  provider  can  work  on 
without  relying  on  referrals  to  specialists. 
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Sample  SMART  Goal  A 


•  "\  am  going  to  take  a  yoga  class  at  the  local 
YMCA  twice  a  week  for  a  month  to  reduce 
stress  and  back  pain." 

•  Feedback:  Right!  This  goal  is  specific, 
measureable,  attainable,  results-focused,  and 
you  can  check  in  with  the  patient  at  the  end  of 
one  month. 


Sample  Un-SMART  Goal  B 


•  "Fm  going  to  detox  tomorrow  and  Fm  never 
going  to  touch  another  pain  killer  again." 

•  Feedback:  While  the  patient  sounds 
motivated,  this  goal  isn't  realistic.  The  patient 
needs  more  information  about  substance-use 
programs  and  how  they  work  (to  understand 
the  commitment).  You  might  need  to  discuss 
medication-assisted  treatment. 
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Sample  SMART  Goal  B 


•  "I'd  like  to  stop  taking  pain  killers  but  don't  think  I  can 
do  that  without  help.  I  want  to  spend  the  next  week 
investigating  substance-use  programs  and  talking  this 
over  with  my  family.  Can  you  point  me  in  the  right 
direction?" 

•  This  is  a  very  realistic  goal.  If  the  patient  devotes  time 
to  investigating  possible  programs,  he  will  better 
understand  what's  involved,  the  level  of  commitment 
required,  and  the  impact  on  him  and  his  family.  He's 
also  giving  himself  a  week  to  investigate,  with  an 
endpoint  (decision  timeframe)  in  mind. 


Wrap  Up 

•  Learners  return  to  the  Care  Planning  Board 
area  to  conclude  this  last  activity 

•  Type  in  at  least  one  well-constructed  SMART 
goal  for  Alex 
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PCP  AVATAR  NAME 


Alex’s  Values 

Alex’s  Symptoms 

-  Alex  c.cire5  ahoui  he  -Pamily. 

-  Alex  care?  about  being 
physically  f-H- 

-  Sleep  disruption 

-  VVeight  gain 

-  Opioid  dependence 

-  Concentration  ^  memory 
esues 

Alex’s  Treatment  Options 

SMART  Goal 

-  Substance  abuse  treatment 

-  Yoga 

-  Coach  boys  soccer  -(-or 
rec  dept 

-  Start  walking  -Por  exercise 

-  \Will  attend  a  weekly 

support  group  at  the  VA 
until  the  next  primary 
care  visit 

End  of  Exploratorium  2 
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